For most of humanity's existence, communication has been incredibly slow. For millennia the only way of transmitting information between two humans was via speech or crude drawings. About 5,000 years ago written language and papyrus increased the transmission distance and bandwidth of human-human communication, but the latency, delivered by hand, was still pretty bad.
Somewhere around 300BC, though—at least according to recorded history—things started to get interesting. Ancient Greece, as described by the historian Polybius, used a technology called hydraulic telegraph to communicate between Sicily and Carthage—a distance of about 300 miles—in the First Punic War.
The system was essentially a slightly higher bandwidth signal fire, with a long unbroken line of humans standing on hilltops with identical telegraph machines. There was still a fair bit of latency, of course, as the humans tweaked the hydraulic levels, but near-speed-of-light electromagnetic radiation was quite a bit faster than papyrus-by-horseback (like the classic example of "driving across the country with a van full of tapes," though, the bandwidth was probably lower).
Again, not much happened for the next 2,000 years, but eventually the whole signal-fire-on-top-of-a-mountain concept was upgraded to formalised semaphore telegraph lines at the end of the 18th century. Semaphore, while a significant improvement over other long-range comms channels, was still hampered by the fact it relied on optics; a human had to be able to physically see the next semaphore tower.
Just a few decades later in 1838, however, the first electrical telegraph was commercialised, and then, well, everything started to accelerate really quickly. A telegraph cable was laid between England and France in 1850; in 1866 the new world was connected to the old; and then by 1872, those amazing Victorians had run cables to India and Australia, connecting up the entire British Empire.
Suddenly and paradoxically the world became both larger and much smaller. Information could travel around the world in a few seconds. Orders could be given, trades could be made, news could be shared. Before the All Red Line a Londoner might've waited weeks or months to hear about antipodean goings-on; once it was completed, yesterday's happenings were in tomorrow's newspaper.
And today, of course, there's no delay at all. We produce, transmit, and consume instantly and incessantly. So it goes.
What a difference a millisecond makes
For better or worse, many tranches of civilisation now rely on sub-second latency. High-frequency trading (HFT), where computers algorithmically interact with the stock market at a very rapid clip, is pointless without an equally quick network connection—you'll just lose out to other HFTs with lower-latency connections. Over time, that could be the difference between making and losing billions of dollars.
Latency matters when it comes to disaster response and national security, too: a few milliseconds might be the difference between a remote sensor sending data back to the lab during a natural disaster or disease outbreak, or a soldier squirting important intelligence back to base before the link is severed. Even social interactions care about latency; I'm sure you've experienced a laggy Skype call before, or had a long-distance phone call bounced off a geosynchronous satellite.As digital technology steadily creeps towards full Homo sapiens integration, latency will become even more important. You probably wouldn't notice the difference between an Internet connection with a latency of 20 and 40 milliseconds, for example; but with a VR headset, those 20ms would be the difference between presence—feeling like you are actually in that simulated world—and nausea. Likewise, as we move towards human-like artificial intelligence and their physical robotic incarnations, a few milliseconds of additional latency can push the robot into the uncanny valley.
But how do we keep latency low? Once upon a time you could halve the latency by two points by simply upgrading the networking gear, or laying a new cable along a more direct route. Today the bigger problem is processing. Your connection to the cloud might be low-latency, but that doesn't help you if the cloud itself takes 100ms to process your query and issue a response; you can forget about using it for AR or another real-time application. Likewise, having a super-fast connection between your VR headset and PC is great, but it won't matter one iota if the graphics card can't draw frames fast enough or the CPU can't keep up with the various positional tracking and physics requirements.
This is why so much effort is going into new methods of processing data. Neuromorphic designs get us closer towards low-latency strong general AI; hyper-specialised processors will push VR and AR towards full immersion and integration; and GPUs and FPGAs in the cloud will backstop everything else that's either too small or too off-the-grid to do its own heavy lifting. Further out—but still not that far away—are crazy technologies like monolithic optoelectronic chips that could increase throughput and reduce latency yet further.
Software, too, will play a big role. Max throughput and ease-of-porting were the main driving forces behind low-level graphics APIs such as Mantle, Vulkan, and Metal, but those same APIs can also be used to reduce the latency of graphics engines if developers so wish. On the cloud side of things, the bigger problem is finding faster and more efficient ways of storing, moving, and processing data—some of that will come down to faster hardware, or moving to in-memory databases, but we'll still need some very clever database management software on top of that.
All of the building blocks are in place for technology that integrates with society, our lives, and probably our bodies, with sub-millisecond latency. What will we do with that technology, though? Or rather, what will that technology do to us? We'll find out in the next few years.