Since the 1990s ancient espionage techniques have become obsolete and 21 st century spies have had to adapt. The old ways have largely been replaced with new methods that take advantage of the new tech; the Internet, cellphones and more powerful and numerous computers along with new software that can do pattern analysis and automatic analysis of photos or video. For spies the most immediate impact of this was that it suddenly became much more difficult for spies to hide their identities and activities. These new tools were most disruptive in police states where it had long been easy to control mass media, communications and free movement. It has taken several decades but some police states developed and implemented ways to deal with the new tech. China is the best example of this and that was no accident. China had the money, the tech and the trained (and loyal) personnel to tame these new technologies and bend them to serve the state rather than enable people to live more freely. Cellphones and the Internet along with the widespread use of security cameras proved capable of creating a surveillance and monitoring system that made it much more difficult to use traditional spies. On the plus side the World Wide Web has made OSINT (Open Source Intelligence) more valuable. OSINT means using information that is available to the public. Even during the Cold War everyone found OSINT useful, if at times tedious, to use. With Internet available much better OSINT can be collected much more quickly.
China led the way by spending billions of dollars to wall off most of its citizens from those many aspects of the World Wide Web that enabled Chinese to find out what was actually happening worldwide and in their own country. China now sells this technology to other nations, or provides it at a big discount for allies who want some modern police state tech to control their own populations. Dictators have found that they cannot just cut their country off from the Internet (as Cuba and North Korea did for a while) because of commercial and government need for Internet access. Moreover the Internet has more ways to leak into a police state than can be blocked.
Perhaps the most notable loss has been the use of “Illegals.” These are spies who do not have diplomatic immunity while posing as embassy personnel), and can be imprisoned, or even executed, if caught. For over a century, the worldwide acceptance of “diplomatic immunity” for embassy personnel was a bonanza for espionage. With this diplomatic immunity you could have some spies in the most restrictive police states. All that new tech has rendered the “legals” much less effective. It was much easier and efficient to steal secrets via the Internet or by tapping into enemy communications wirelessly. Illegals are costly and more vulnerable because of the surveillance state and better search tools.
Most countries used a lot of diplomatic personnel, without diplomatic immunity, as “illegals” and even this practice has been crippled by the new tech. But the most important illegals were those who were living in a foreign country pretending to be locals, or migrants from some friendly nation. The Russians were very good at creating convincing “legends” (fake identities and back stories) for their illegals. During the Cold War, the Russians were so good that they were rumored to have special boarding schools where promising Russian children were sent to learn how to speak and act like an American (or German, or Briton or Brazilian or whatever). This was mostly fantasy, but there were schools that taught the customs of foreign nations, and language institutes where illegals could have their accent tweaked to eliminate all trace of its Russian origin.
Russia would also recruit spies in third countries, and train them to be illegals in another nation, like the United States (where there were always a lot of migrants.) All these illegals were employees of the KGB (the Russian CIA/FBI), had KGB ranks, and, if they stayed alive and were successful, would eventually retire to a comfortable life on their KGB pension. Many did so, although dozens were caught and served long jail terms. A few were exchanged for U.S. spies in Russian jails. Some illegals switched sides, and had to worry about KGB death squads until the Soviet Union collapsed in 1991.
All those KGB schools, and most of the world class KGB expertise, disappeared with the Soviet Union after 1991. The ten illegals caught in the United States in 2010 were strictly amateurs, although they had some training and were employees of the FSB (the much smaller Russian successor to the KGB). But their language and cultural training were not up to KGB Cold War standards. Neither were their espionage skills. All ten were quickly detected and put under surveillance by the American FBI for years in the hope that they could learn as much as possible about how the FSB operated, before rounding the illegals up. This crew were arrested when one of them apparently began suspecting that they were being watched, and reported this back to Russia. The FBI was indeed watching, and managed to arrest ten of the eleven Russian illegals they were monitoring. The eleventh spy may have been a double agent, as the Russians said little about him. The rest returned to Russia and were declared heroes.
The FBI, obviously, did not release a lot of details, because they are likely other Russian illegals being watched or sought. Some of these may not been confirmed as illegals, or may have been called back to Russia. Details on that sort of thing will be revealed in the future. Needless to say, all this espionage continues, much as it did when the Soviet Union collapsed. During the 1990s, a lot of suddenly (or potentially) unemployed KGB personnel (including legals, and officials back in Russia), offered to sell information to the CIA and FBI. Many of these deals were consummated, and Russia’s formidable Cold War espionage network suffered a lot of damage in the 1990s. Since 2000 Russia has been rebuilding, but it won’t be the same. Now that we know how extensive the KGB espionage network was (due to all those 1990s turncoats), it’s unlikely anyone else will have the resources, or ignorance in the West, to pull it off.
Some of the damage Russian espionage suffered in the 1990s was kept secret for a while so it could be fully exploited. One example we know of was revealed in 2014 when Britain gave the public access to most of the secret KGB files obtained in 1992 when Britain smuggled Vasili Mitrokhin, a senior KGB official in charge of the KGB archives, out of Russia along with thousands of KGB documents Mitrokhin had copied and hidden for over a decade. Mitrokhin had offered the files to the U.S. first, but was turned away. Then he tried the British, who immediately recognized the opportunity and not only got Mitrokhin out of Russia along with his files, but set him up in comfortable, and anonymous, retirement in Britain until he passed away in 2004 at age 81.
The Mitrokhin files were a goldmine of information and a disaster for Russian intelligence. This apparently contributed to the current extreme anti-Western hostility shown by senior Russian officials who used to be KGB officers. This includes Vladimir Putin, who has run Russia since 1999 and brought a lot of his former KGB cronies into the government. The Mitrokhin files and the presence of Mitrokhin in Britain was kept secret for over a decade so that the data in those files could be exploited. In addition to lists of most KGB Cold War operations (including many Western intel agencies were not even aware of) there were also the names of over a thousand active and “sleepers” (agents that often spend most of their time doing nothing, until activated from time-to-time for some simple, but essential, mission) agents operating in the West and the East European nations that were once referred to (until 1989) as Russian “satellites.”
Before the 1990s were over the Russians figured out what had happened and they were not happy about it. Mitrokhin had spent his career in the KGB archives and eventually became the guy in charge. For an espionage agency, having a leak in the archives is the worst possible nightmare. Mitrokhin had become disenchanted with the Soviet Union in the late 1970s and risked his life for over a decade sneaking out archive documents, copying them by hand and then returning the originals. But he never dared offer them to a foreign government because a man in his position was well guarded and constantly watched. No one ever caught on to the document duplication and to Mitrokhin that in itself was a major achievement. Then the Soviet Union suddenly ceased to exist in 1991. People like Mitrokhin, with access to secret opinion surveys and more accurate data on economic performance and how inept the national leadership had become, saw this coming, eventually. Mitrokhin also noticed how morale and performance collapsed after the Soviet Union was gone and that gave him the opportunity, and confidence, to make a break for it.
His disappearance was not unusual because a lot of KGB officers had been disenchanted with their communist government but did nothing about it until the Soviet Union collapsed. Many left Russia to find their fortune elsewhere. Some were selling KGB secrets and many of these were later hunted down and killed. Others stayed and were running the new Russia by the late 1990s. The KGB had always recruited the “best and the brightest” and rewarded them well for performance and loyalty. Traitors were executed but these were few because those who applied to join the KGB knew what they were getting into and were content to have interesting work and lots of fringe benefits. This included immunity from arrest except by other KGB officers.
In addition to the names of agents and descriptions of operations the Mitrokhin files also contained lists of secret weapons, explosives and equipment caches hidden in the West. These were to be used by sleepers in emergencies or in the event of war with the Soviet Union. Many of these caches were quietly visited and cleaned out. In some cases the stuff was already gone, indicating that some sleepers saw the end of the Cold War as an emergency and considered the caches they knew about as a form of severance pay. The Mitrokhin files also contained interesting details on the personalities and effectiveness of the foreign agents. This required some existing histories of known Russian spies to be revised.
In the West the roundup of former Soviet spies and sleeper agents quietly began in the early 1990s. Initially this was done using a convincing “legend” (reasonable explanations of how the spies were identified without help from the KGB archives) for each arrest. If the Russians had figured out the extent of the Mitrokhin files, or that they even existed, the world would have reached all this Soviet era spies and most would have fled back to Russia or gone dark (cut off communication with the new Russian spy agencies) assumed new identities and backgrounds and tied to disappear in the West (where life was better).
China had observed the collapse of the traditional espionage techniques in the 1990s and the impact of cellphones and the Internet on information, censorship and espionage. The Chinese were quick to exploit the new tech and the new opportunities. Worst of all, the generally complacent West, where all this new tech came from, was slow to catch on and are now furiously playing catch up and trying to assess the extent of the damage already done. Russia has not got the resources in manpower, tech and money that China has and can only watch with envy as China takes the place of Russia as the greatest practitioner of espionage.