"As today 25 March 2014, PayPal is launching a new survey program. All customers are welcome to participate this survey. The survey will take 5 minutes and for your effort and understanding PayPal will select most of the customers that takes this survey and reward them with £25.00." 

This is the usual sort of ungrammatical nonsense that pours into our email inboxes every day, asking the recipient to click on a malware-containing attachment that, hopefully, most of us know to ignore.

But in the name of security research, some are turning the tables on the daily deluge of maliciousness. They set up what are known in the industry as "honeypots," fake but genuine-looking internet servers that are used by security teams to attract attackers in order to understand their latest techniques and the hottest malicious software doing the rounds.

Earlier this year, in the black heart of the City of London, Europe's financial capital, I talked to a group of penetration testers (ethical hackers who poke holes in their customers' systems to figure out where they are weakest), who agreed to create some new honeypots and demonstrate their use for me. I wanted to understand more about how honeypots were built, and whether we could glean any patterns if we added fresh traps in new locations.

Honeypots are normally created on virtual private servers—rentable places to host things on the internet. Once you've bought your plot of land for a couple of quid, you download honeypot software; in our case, we used programs known as Dionaea and Kippo. This process is essentially like installing a new operating system onto a dumb machine, and creates what appears to hackers to be a genuinely vulnerable server. In reality, none of the features of the systems work, but they look real enough. 

If things go well, hackers find these servers, scan for weaknesses, and break into them for whatever purposes they choose. It's like hosting an open house for robbers—except when they find there's nothing inside, they set up shop and start trying to steal things or create a base for storing criminal tools. All the while, as owners of the house, the security workers watch over them, record their malicious moves, and stop them actually doing anything nasty to people on the internet. 

Andy Swift, a pentester who set up the honeypots. Image: Andy Swift

Ideally, these honeypots should be placed in parts of the world that are rife with digital criminality, so that those running them get a bigger data set of attacks to better understand what hackers are up to. Right at the start of March, Andy Swift, one of the pentesters from consultancy firm Hut3, bought some virtual private servers in four such areas: China, Russia, Kazakhstan, and Singapore.

Within seconds of the honeypots being live, attackers flocked to our dummy servers, their scanning tools quickly able to identify the vulnerable machines. Most attackers will not peruse the internet trying to identify a multitude of vulnerabilities. Instead, they will repeatedly probe with tools like Masscan for weak machines on which their devised exploits will work. "This is much quicker and much more effective for launching a specific attack," said Swift. As honeypot machines contain all kinds of vulnerabilities, they will entice many different kinds of hacker. 

After just five minutes, the Chinese honeypot had attracted 19 separate hackers, who had tried 1,000 different ways to break into the server and exploit it. By the following day, the number of malicious IP addresses connecting to the server hit 431. That's a 22-fold increase over 24 hours. "Its startup benchmarks are much higher than any we have seen before," said Swift, who has experience creating honeypots as part of his company's Cyber Intelligence Network.

This would be a running theme for the honeypot project as a whole: China was buzzing with illicit activity. At the end of March, 3,879 unique attacks had been recorded on the Chinese honeypot, and between them they'd lumped 36 different kinds of malware on the server. Most attacks (2003 in total) were sourced back to China too. On both the Singapore and Kazakhstan machines, China was also the source of most attacks, with 1,266 out of 4,179 and 425 out of 1,481, respectively.

This doesn't necessarily mean the attackers were all Chinese; they could simply have been using rented machines in China. But it nevertheless provides an idea of just how much attack traffic is going through the Asian superpower's systems. The map below, which was fed with data from across all of Swift's honeypots, the new ones included, shows the US and China as the two busiest locations for nefarious digital activity.

The chart below shows how far ahead China is in terms of the number of attacks it sees. It also shows the impact of our newly added honeypots on the number of attacks and "threat level," as March shows a noticeable rise in incidents. Again, this is largely due to activity from China.

Andy managed to video one attack passing through our Chinese honeypot, which was sourced back to China too. Here we see the attacker logging on to the honeypot through a terminal, via the server's file transfer protocol (FTP) service. (FTP stores are simply spaces where people can keep their files and send them easily across the internet).

The attacker guesses a username of "root" and a password of "password," which doesn't show up on the terminal in case any real-world snoops happen to be passing by the screen. The hacker doesn't know that the honeypot will accept any password it's given. (Note the attacker commands are seen next to "ftp>" and we've obscured the IP address.)

You'll see them first type "dir," a command asking for a directory that lists everything sitting on the honeypot. The directory is evidently empty, so the attacker then runs the "help" command to get the full list of possible actions they can take on the fake FTP server. They then attempt to upload malware, using the "put" command to transfer a file called "sdklfsdlk.exe." That file turns out to be a basic Windows keylogger that captures users' every key entry and transfers it back to the attacker's computer. This could be used in a variety of scenarios, but attackers are particularly fond of operating keyloggers to figure out people's online bank account passwords, or any logins going through the infected machine.

But then the attacker shows their hacking skills are somewhat rudimentary, according to Swift. "Firstly, they have uploaded a bit of Windows malware to a UNIX operating system, which will never work," he said. "Secondly, they then attempt to run the Windows malware from the UNIX-based FTP server. Confused, the attacker then types 'help' again and exits the server, leaving the malware there, and will likely head off to try their next target." 

This screenshot is of a table that keeps tabs on all the different honeypots live. You can see under "geolocation" that several attacks are based in China.

Over in Russia, which we had expected to be at least as busy, our honeypot was comparatively quiet, with just 694 attacks over the whole month—less than Singapore and Kazakhstan, too. But the country, known as a hotbed for malicious software, did see the most interesting, unique malware going through its systems. Two pieces of malicious software in particular caught our eye. The first, called Atak, was an email worm, a kind of malware that distributes copies of itself in files attached to automated email messages. 

Once they've infected a machine, these worms will search for email accounts to send more messages and so continue to proliferate. "Atak looks for other machines to spread itself to on the local network and also queries multiple SMTP servers [servers for sending and receiving email], in particular those belonging to Google (Gmail)," Swift explained. "It typically uses this to send out spam email or emails containing itself so it can spread to other machines."

The second was named Virut, a well-known piece of malware that attackers use to set up networks of infected machines, called botnets, for pilfering data and sending out spam. Early last year, the operator of the Polish ".pl" top-level domain, Naukowa i Akademicka Sieć Komputerowa (NASK), took control of domains used by the Virut controllers in an attempt to shut it down. People suspected the malware would survive, as some of its master servers were based in Russia—and our honeypot proved Virut botnets are still very much alive.

"This bit of malware comes in a number of of variants. It's pretty interesting in that it has entry-point obscuring abilities: basically, it can also take over and inject itself into any executable file it feels like," said Swift. "Aside from this, the malware itself, once run, connects back to an IRC [internet relay chat] server where the distributor can control and issue arbitrary commands to the malware." That basically means an infected machine is in the hands of the attacker.

The beautiful thing about Swift's Russian honeypot was its ability to play with the hacker, rather than just record their movements. In the video below, you can see how a hacker gets lured in and then taunted by an emoticon bird.

Here, a hacker logs in and types in the "wget" command, which retrieves files or web pages from a web server, in this case a file named f.tgz. Once downloaded on to the honeypot machine, the attacker then decompresses the file, which contains a load of scripts that could potentially be malicious tool kits for launching further attacks.

They then change directory using the "cd" command, using a directory called "fresh," as they attempt to run one of their malicious files, which is simply named "a." This is when the system messes with the attacker, refusing to run the malicious script. Instead, the terminal displays a text owl, accompanied by the words "O RLY?" The attacker evidently becomes befuddled, typing in a "yes" command, to which the owl responds: "NO WAI!" Indeed.

Halfway through the month-long project, Swift decided he wanted to catch some spammers in action. He set up what's known as an open SMTP relay—essentially a server that can easily be hacked to send out reams of spam. Our server would allow a hacker to break in and set up a spam campaign, but it would prevent those irritating messages from actually hitting people's inboxes. After just two hours, a malicious actor acquired access to the server and was trying to send out messages offering some "New Nike Trainers." As tends to happen with spam, there was little else of interest other than more attempts to peddle knock-off goods.

Despite all the apparent opportunities our honeypots appeared to offer attackers, with the myriad security holes they had left wide open, the majority of hacks were simply interested in acquiring space on the servers so they could offer anonymous, cheap phone services to people on hacker forums. To do this, an attacker checks to see whether the server's Session Initiation Protocol (SIP) port, used for voice and video calls between IP addresses, is open.

Once they have control over the service, they often sell them to phone spammers on illicit markets. By using a hacked server, scammers will effectively cover their tracks when making irritating calls. In the worst case scenario, the SIP port would be linked to a genuine VoIP service and that would be abused to make calls to a premium rate number controlled by the attacker. The crooks would get a chunk of the proceeds from each call, so it's an easy way for them to make money. From our honeypots, which didn't allow calls to actually connect, it was clear hackers saw a lot of value in this area.

Also hugely common were attacks on an old, but still much exploited vulnerability in Microsoft Windows, known as MS08-67. It's a known weak spot in Windows 2000 up to XP, but while it's been patched numerous times, Swift said a lot of companies still haven't got it covered. "This is the go-to exploit in pentesting," he said. "We see it all the time. If you see this exploit in Windows 2003, it's a sure fire thing you're going to access that machine." (You can see the rate at which different operating systems were targeted in one of the graphs above, "Operating System Attacks.")

Now that our month-long project is over, what it really made apparent is that many of the internet's old problems aren't solved. SIP attacks and spam continue to thrive, despite being common knowledge for years. Companies are still failing to patch systems to prevent attacks, despite the assistance on offer. China and Russia in particular continue to host a startling amount of criminal activity. 

An obvious question then arises: how do honeypot projects, which have been prevalent for years amongst researchers, actually help cut digital criminality? Many of the world's major security companies, from Symantec to Kasperksy, run them. Yet cybercrime continues to rise. Data released by Symantec in April showed that in 2013, the number of data breaches grew 62 percent over the previous year, exposing approximately 552 million people's identities.

Honeypots may not be directly preventing online crime, but it's clear they are of value to pentesters. That's because pentesters' jobs are not about ending malicious activity, but working with their paymasters to ensure their infrastructure is as safe from attack as possible. The research derived from honeypots is only one small part of that process.

For now, the digital criminals we came into contact with will likely go on to find and exploit genuinely vulnerable systems. Even though we had fun watching hackers fail, and being repeatedly mocked by an owl, it's hard to come away feeling anything but concerned about the security of the internet at large.