Network discovery using UDP Broadcast (Java)

The Problem

I have a Java server and a Java client running on the same network and the applications are not to be used outside a private network (not over internet). So I used a static IP for the server, but what if I deploy my application? What if the network changes? That means I’ll lose my connection to the server and I’ll have to change the IP on the client side again. Now that would be stupid. I want the client to “discover” the server on the network and connect with it.

The Solution

Using UDP packets and broadcasting them! This technique however is not optimal, but as long as we stay in one network this shouldn’t be a problem. UDP packets however are fairly easy to work with. So let’s get started.

Still here? Let’s do this!

Server implementation

First, Let’s create the Java Singleton class that will execute the code on the server-side. This will be multi-threaded of course,  so we’ll also implement “Runnable”. When we implement Runnable, we also have to override the Run method.

public class DiscoveryThread implements Runnable {

  public void run() {

  public static DiscoveryThread getInstance() {
    return DiscoveryThreadHolder.INSTANCE;

  private static class DiscoveryThreadHolder {

    private static final DiscoveryThread INSTANCE = new DiscoveryThread();


Ok, let’s think about this. What do we have to do?

  1. Open a socket on the server that listens to the UDP requests. (I’ve chosen 8888)
  2. Make a loop that handles the UDP requests and responses
  3. Inside the loop, check the received UPD packet to see if it’s valid
  4. Still inside the loop, send a response to the IP and Port of the received packet

That’s it on the server side.

Now, we’ll translate this into code.

DatagramSocket socket;

  public void run() {
    try {
      //Keep a socket open to listen to all the UDP trafic that is destined for this port
      socket = new DatagramSocket(8888, InetAddress.getByName(""));

      while (true) {
        System.out.println(getClass().getName() + ">>>Ready to receive broadcast packets!");

        //Receive a packet
        byte[] recvBuf = new byte[15000];
        DatagramPacket packet = new DatagramPacket(recvBuf, recvBuf.length);

        //Packet received
        System.out.println(getClass().getName() + ">>>Discovery packet received from: " + packet.getAddress().getHostAddress());
        System.out.println(getClass().getName() + ">>>Packet received; data: " + new String(packet.getData()));

        //See if the packet holds the right command (message)
        String message = new String(packet.getData()).trim();
        if (message.equals("DISCOVER_FUIFSERVER_REQUEST")) {
          byte[] sendData = "DISCOVER_FUIFSERVER_RESPONSE".getBytes();

          //Send a response
          DatagramPacket sendPacket = new DatagramPacket(sendData, sendData.length, packet.getAddress(), packet.getPort());

          System.out.println(getClass().getName() + ">>>Sent packet to: " + sendPacket.getAddress().getHostAddress());
    } catch (IOException ex) {
      Logger.getLogger(DiscoveryThread.class.getName()).log(Level.SEVERE, null, ex);

A few notes; If you want to use strings as commands (like I do in this example), you have to trim the string before comparing it.

There, that’s it for the server.

Client implementation

Now we have to write the code for the client. Again, let me sketch how we are going to work.

  1. Open a socket on a random port.
  2. Try to broadcast to the default broadcast address (
  3. Loop over all the computer’s network interfaces and get their broadcast addresses
  4. Send the UDP packet inside the loop to the interface’s broadcast address
  5. Wait for a reply
  6. When we have a reply, check to see if the package is valid
  7. When it’s valid, get the package’s sender IP address; this is the server’s IP address
  8. CLOSE the socket! We don’t want to leave open random ports on someone else’s computer

On a side note, we don’t close the socket on the server because the server will receive and send UPD packets until the server is closed. Closing the socket on the server means that we won’t be able to discover it any more.

Wow, that was quite a lot. Now let’s put this into code!

// Find the server using UDP broadcast
try {
  //Open a random port to send the package
  c = new DatagramSocket();

  byte[] sendData = "DISCOVER_FUIFSERVER_REQUEST".getBytes();

  //Try the first
  try {
    DatagramPacket sendPacket = new DatagramPacket(sendData, sendData.length, InetAddress.getByName(""), 8888);
    System.out.println(getClass().getName() + ">>> Request packet sent to: (DEFAULT)");
  } catch (Exception e) {

  // Broadcast the message over all the network interfaces
  Enumeration interfaces = NetworkInterface.getNetworkInterfaces();
  while (interfaces.hasMoreElements()) {
    NetworkInterface networkInterface = interfaces.nextElement();

    if (networkInterface.isLoopback() || !networkInterface.isUp()) {
      continue; // Don't want to broadcast to the loopback interface

    for (InterfaceAddress interfaceAddress : networkInterface.getInterfaceAddresses()) {
      InetAddress broadcast = interfaceAddress.getBroadcast();
      if (broadcast == null) {

      // Send the broadcast package!
      try {
        DatagramPacket sendPacket = new DatagramPacket(sendData, sendData.length, broadcast, 8888);
      } catch (Exception e) {

      System.out.println(getClass().getName() + ">>> Request packet sent to: " + broadcast.getHostAddress() + "; Interface: " + networkInterface.getDisplayName());

  System.out.println(getClass().getName() + ">>> Done looping over all network interfaces. Now waiting for a reply!");

  //Wait for a response
  byte[] recvBuf = new byte[15000];
  DatagramPacket receivePacket = new DatagramPacket(recvBuf, recvBuf.length);

  //We have a response
  System.out.println(getClass().getName() + ">>> Broadcast response from server: " + receivePacket.getAddress().getHostAddress());

  //Check if the message is correct
  String message = new String(receivePacket.getData()).trim();
  if (message.equals("DISCOVER_FUIFSERVER_RESPONSE")) {
    //DO SOMETHING WITH THE SERVER'S IP (for example, store it in your controller)

  //Close the port!
} catch (IOException ex) {
  Logger.getLogger(LoginWindow.class.getName()).log(Level.SEVERE, null, ex);

I’ve given you pretty much all of the code, so it shouldn’t be easy to implement. Don’t forget to run your DiscoveryThread!

Thread discoveryThread = new Thread(DiscoveryThread.getInstance());

Handling network timeouts in Java

When writing network applications in a stable and controlled environment, it is easy to forget what the real world is like. Slow connections, traffic build ups, or power interruptions can cause network connections to stall or even die. Few programmers take the time to detect and handle network timeouts, but avoiding the problem can cause client applications to freeze, and for threads in servers to block indefinitely. There are, however, easy ways to handle network timeouts. In this article, I’ll present two techniques for overcoming the problem of network timeouts – threads and setting a socket option for timeouts.


Handling timeouts is one of those tasks that people generally leave to the last moment. In an ideal world, we’d never need them. In an intranet environment, where most programmers do their development, it’s almost always unnecessary. However, on the Internet, good timeout handling is critical. Badly behaved clients may go offline and leaving server threads locked, or overloaded servers may stall, causing a network client to block indefinitely for input. For these reasons, it is necessary to detect and handle network timeouts.

I’ve identified two relatively simple solutions to the problem of handling network timeouts. The first solution involves the use of second thread, which acts as a timer. This solution results in a slight increase in complexity, but is backwards compatible with JDK1.02. The second solution is by far, much simpler. It takes only a few lines of code, by setting a socket option, and catching a timeout exception. The catch is, that it requires a JDK1.1 or higher virtual machine.

Solution One : Using a timer thread

Many network software (particularly servers), are written as multi-threaded applications. However, a client can also use multiple threads. One solution to handling network timeouts is to launch a secondary thread, the ‘timer’, and have it cancel the application gracefully if it becomes stalled. This prevents end users from becoming confused when the application stalls – a good error message will at least let them know the cause of the problem.

Listing One shows a Timer, which can be used in networking applications to handle timeouts gracefully. Once the timer is started, it must be reset regularly, such as when data is returned by a server. However, if the thread that reads from a remote network host becomes stalled, the timer will exit with an error message. For those who require a custom handler, the timeout() method may be overridden to provide different functionality.

Listing One –

  * The Timer class allows a graceful exit when an application
  * is stalled due to a networking timeout. Once the timer is
  * set, it must be cleared via the reset() method, or the
  * timeout() method is called.
  * <p>
  * The timeout length is customizable, by changing the 'length'
  * property, or through the constructor. The length represents
  * the length of the timer in milliseconds.
  * @author	David Reilly
public class Timer extends Thread
	/** Rate at which timer is checked */
	protected int m_rate = 100;
	/** Length of timeout */
	private int m_length;

	/** Time elapsed */
	private int m_elapsed;

	  * Creates a timer of a specified length
	  * @param	length	Length of time before timeout occurs
	public Timer ( int length )
		// Assign to member variable
		m_length = length;

		// Set time elapsed
		m_elapsed = 0;

	/** Resets the timer back to zero */
	public synchronized void reset()
		m_elapsed = 0;

	/** Performs timer specific code */
	public void run()
		// Keep looping
		for (;;)
			// Put the timer to sleep
			catch (InterruptedException ioe) 

			// Use 'synchronized' to prevent conflicts
			synchronized ( this )
				// Increment time remaining
				m_elapsed += m_rate;

				// Check to see if the time has been exceeded
				if (m_elapsed > m_length)
					// Trigger a timeout


	// Override this to provide custom functionality
	public void timeout()
		System.err.println ("Network timeout occurred.... terminating");

To illustrate the use of the Timer class, here is a simple TCP client (Listing Two) and server (Listing Three). The client sends a text string across the network, and then reads a response. While reading, it would become blocked if the server stalled, or if the server took too long to accept the connection. For this reason, a timer is started before connecting, and then reset after each major operation. Starting the timer is relatively simple, but it must be reset after each blocking operation or the client will terminate.

// Start timer
Timer timer = new Timer(3000);

// Perform some read operation

// Reset the timer

The server is relatively simple – it’s a single-threaded application, which simulates a stalled server. The server reads a response from the client, and echoes it back.To demonstrate timeouts, the server will always “stall” for a period of twenty seconds, on every second connection. Remember though, in real life, server timeouts are entirely unpredictable, and will not always correct themselves after several seconds of delay.

Listing Two


  * SimpleClient connects to TCP port 2000, writes a line
  * of text, and then reads the echoed text back from the server.
  * This client will detect network timeouts, and exit gracefully,
  * rather than stalling.
  * Start server, and the run by typing
  *  java SimpleClient
public class SimpleClient
	/** Connects to a server on port 2000,
	    and handles network timeouts gracefully 
	public static void main (String args[]) throws Exception
		System.out.println ("Starting timer.");
		// Start timer
		Timer timer = new Timer(3000);

		// Connect to remote host
		Socket socket = new Socket ("localhost", 2000);
		System.out.println ("Connected to localhost:2000");

		// Reset timer - timeout can occur on connect

		// Create a print stream for writing
		PrintStream pout = new PrintStream ( 
			socket.getOutputStream() );

		// Create a data input stream for reading
		DataInputStream din = new DataInputStream( 
			socket.getInputStream() );

		// Print hello msg
		pout.println ("Hello world!");

		// Reset timer - timeout is likely to occur during the read

		// Print msg from server
		System.out.println (din.readLine());

		// Shutdown timer

		// Close connection

Listing Three


  * SimpleServer binds to TCP port 2000, reads a line
  * of text, and then echoes it back to the user. To
  * demonstrate a network timeout, every second connection
  * will stall for twenty seconds.
  * Start server by typing
  *  java SimpleServer
public class SimpleServer extends Thread
	/** Shall we stall? flag */
	protected static boolean shallWeStall = false;

	/** Socket connection */
	private Socket m_connection;

	  * Constructor, accepting a socket connection
	  *	@param	connection	Connection to process
	public SimpleServer (Socket connection)
		// Assign to member variable
		m_connection = connection;

	/** Starts a simple server on port 2000 */
	public static void main (String args[]) throws Exception
		ServerSocket server = new ServerSocket (2000);

		for (;;)
			// Accept an incoming connection
			Socket connection = server.accept();

			// Process in another thread
			new SimpleServer(connection).start();

	/** Performs connection handling */
	public void run()
			DataInputStream din = new DataInputStream (
				m_connection.getInputStream() );

			PrintStream pout = new PrintStream (
				m_connection.getOutputStream() );

			// Read line from client
			String data = din.readLine();

			// Check to see if we should simulate a stalled server
			if (shallWeStall)
				// Yes .. so reset flag and stall
				shallWeStall = false;

					Thread.sleep (20000);
				} catch (InterruptedException ie ) {}
				// No.... but we will next time
				shallWeStall = true;

			// Echo data back to clinet
			pout.println (data);

			// Close connection
		catch (IOException ioe) 
			System.err.println ("I/O error");

Solution Two : Simplified timeout handling with socket options

A significantly easier solution to handling network timeouts is to set a socket option. Socket options allow programmers greater control over socket communication, and are supported by Java as of JDK1.1. One socket option in particular, SO_TIMEOUT, is extremely useful, because it allows programmers to specify an amount of time that a read operation will block for, before generating, which can be easily caught to provide a timeout handler.

We can specify a value for SO_TIMEOUT, by using the setSoTimeout() method. This method is supported by,, and This is important, as it means we can handle timeouts in both the client and the server. The setSoTimeout accepts as its sole parameter an int, which represents the number of milliseconds an operation may block for. Settings this value to one thousand will result in timeouts after one second of inactivity, whereas setting SO_TIMEOUT to zero will allow the thread to block indefinitely.

// Set SO_TIMEOUT for five seconds

Once the length for the timeout is set, any blocking operation will cause an InterruptedIOException to be thrown. For example, a DatagramSocket that failed to receive packets when the receive() method is called, would throw an exception. This makes it easy to separate our timeout handling code from the task of communicating via the network.

The next example, Listing Four, shows another multi-threaded echo server with timeout support. It limits the number of connections it can support (currently set to two), and rejects further connections. However, if a client fails to send data, a server thread will become blocked, and the number of available connections will be reduced. With larger servers, and hundreds or thousands of connections per hour, blocked threads become a significant problem, and can lead to denial of service. This example shows how to detect a timeout, and to disconnect gracefully. By using socket_options, and catching exceptions, it’s easier to shut down an individual thread.

To test the echo server, you can use the telnet command (available on both Unix/Wintel systems), and connect to port 2000 of your local machine. Type a few characters of text, and then watch what happens after thirty seconds of inactivity. The server will automatically disconnect the telnet client, freeing up the connection for another user.

Listing Four


  * EchoServer offers an echo service to multiple clients.
  * The echo service is limited to a set number of connections,
  * to prevent server over-load. It also includes timeout handling
  * code to prevent server threads from blocking if a client is
  * stalled.
public class EchoServer extends Thread
	/** Connection to client */
	private Socket m_connection;

	/** Number of connections */
	private static int number_of_connections = 0;

	/** Maximum number of connections */
	private static final int max_connections = 2;

	/** Port to bind to */
	private static final int service_port = 2000;

	/** Timeout length */
	private static final int timeout_length = 30000;

	  * Creates a new instance of EchoServer thread, to
	  * service the specified client connection.
	  * @param connection	Connection to service
	public EchoServer (Socket connection)
		// Assign to member variable
		m_connection = connection;

		// Set a timeout of 'timeout_length' milliseconds
			m_connection.setSoTimeout (timeout_length);
		catch (SocketException se)
			System.err.println ("Unable to set socket option SO_TIMEOUT");

		// Increment number of connections

	public void run()
			// Get I/O streams
			InputStream  in = m_connection.getInputStream();
			OutputStream out= m_connection.getOutputStream();

				for (;;)
					// Echo data straight back to client
			catch (InterruptedIOException iioe)
				System.out.println ("Timeout occurred - killing connection");
		catch (IOException ioe)
			// No code required - thread will terminate at end
			// of the run method

		// Decrement the number of connections

	public static void main(String args[]) throws Exception
		// Bind to a local port
		ServerSocket server = new ServerSocket (service_port);

		for (;;)
			// Accept the next connection
			Socket connection = server.accept();

			// Check to see if maximum reached
			if (number_of_connections > max_connections-1)
				// Kill the connection
				PrintStream pout = new PrintStream (connection.getOutputStream());
				pout.println ("Too many users");
				// Launch a new thread
				new EchoServer(connection).start();


All good network applications will include timeout detection and handling. Whether you’re writing a client, and need to detect a wayward server, or writing a server and need to prevent stalled connections, timeout handling is a critical part of error handling. For those who require backwards compatibility with JDK1.02, timers may be used to detect stalled connections. The most preferable solution though, is to use socket options, and to provide an exception handler for This reduces the amount of code required to handle timeouts, and makes for a cleaner design.

Gang of Four (GoF) Design Pattern

Gang of Four (GoF) Patterns are 23 main software design patterns providing recurring solutions to common problems in software design. They were developed by  Erich Gamma, Richard Helm, Ralph Johnson and John Vlissides, often referred to as the Gang of Four.

We have started this series to explain each of these patterns with a real world example and how to develop and use them in Java.

Creational Design Pattern:

This is all about object creation in a form that they can follow the architecture principle of loose coupling, encapsulation and abstraction. This pattern can provide a great deal of flexibility about which objects are created, how those objects are created, and how they’re initialized. And the most important thing is that the client remains completely unaware of overall object creation process.

 Structural Design Pattern:

This pattern is used to form larger object structures from many different objects.  They show you how to use composition to join different pieces of a system together in a flexible and extensible fashion. This pattern help you guarantee that when one of the parts changes, the entire structure doesn’t need to change. They also show you how to recast pieces that don’t fit into pieces that do fit.

 Behaviour Design Pattern:

This pattern is used to manage relationships, interaction, algorithms and responsibilities between various objects. It also follows the abstraction principle whereby it abstracts the action. This pattern focuses on the interaction between the cooperating objects in a manner that the objects are communicating while maintaining loose coupling.

Creational Pattern Structural Patterns Behavioral Patterns
Abstract Factory Adapter Chain of Responsibility
Builder Bridge Command
Factory Method Composite Interpreter
Prototype Decorator Iterator
Singleton Facade Mediator
Flyweight Memento
Proxy Observer
Template Method

The below diagram provides a clear view of the relationship between the above design patterns mentioned.

Design Pattern Relationship

Types of Creational Design Patterns (as defined by Wikipedia):
Abstract Factory groups object factories that have a common theme.
Builder constructs complex objects by separating construction and representation.
Factory Method creates objects without specifying the exact class to create.
Prototype creates objects by cloning an existing object.
Singleton restricts object creation for a class to only one instance.

Types of Structural Design Pattern (as defined by Wikipedia):
Adapter allows classes with incompatible interfaces to work together by wrapping its own interface around that of an already existing class.
Bridge decouples an abstraction from its implementation so that the two can vary independently.
Composite composes zero-or-more similar objects so that they can be manipulated as one object.
Decorator dynamically adds/overrides behaviour in an existing method of an object.
Facade provides a simplified interface to a large body of code.
Flyweight reduces the cost of creating and manipulating a large number of similar objects.
Proxy provides a placeholder for another object to control access, reduce cost, and reduce complexity.

Types of Behavioral Design Pattern (as defined by Wikipedia):
Command creates objects which encapsulate actions and parameters.
Interpreter implements a specialized language.
Iterator accesses the elements of an object sequentially without exposing its underlying representation.
Mediator allows loose coupling between classes by being the only class that has detailed knowledge of their methods.
Memento provides the ability to restore an object to its previous state (undo).
Observer is a publish/subscribe pattern which allows a number of observer objects to see an event.
State allows an object to alter its behavior when its internal state changes.
Strategy allows one of a family of algorithms to be selected on-the-fly at runtime.
Template method defines the skeleton of an algorithm as an abstract class, allowing its subclasses to provide concrete behavior.
Visitor separates an algorithm from an object structure by moving the hierarchy of methods into one object.

Things to know about Stack & Heap

The JVM divided the memory into following sections.

  1. Heap
  2. Stack
  3. Code
  4. Static

This division of memory is required for its effective management.

The code section contains your bytecode.

The Stack section of memory contains methods, local variables and reference variables.

The Heap section contains Objects (may also contain reference variables).

The Static section contains Static data/methods.

Of all of the above 4 sections, you need to understand the allocation of memory in Stack & Heap the most, since it will affect your programming efforts

difference between instance variables and local variables

Instance variables are declared inside a class but not inside a method

class Student{
int num; // num is  instance variable
public void showData{}

Local variables are declared inside a method including method arguments.

public void sum(int a){
int x = int a +  3;
// a , x are local variables</strong>

the video demonstrates how memory is allocated in stack & heap.

Please be patient. The Video will load in some time. If you still face issue viewing video click here

Points to Remember:

  • When a method is called , a frame is created on the top of stack.
  • Once a method has completed execution , flow of control returns to the calling method and its corresponding stack frame is flushed.
  • Local variables are created in the stack
  • Instance variables are created in the heap & are part of the object they belong to.
  • Reference variables are created in the stack.

Point to Ponder: What if Object has a reference as its instance variable?

public static void main(String args[]){ A parent = new A(); //more code } class A{ B child = new B(); int e; //more code } class B{ int c; int d; //more code }

In this case , the reference variable “child” will be created in heap ,which in turn will be pointing to its object, something like the diagram shown below.

When to use “static” keyword in Java

The static keyword can be used in 3 scenarios

  1. static variables
  2. static methods
  3. static blocks of code.

Lets look at static variables and static methods first.

static variable

  • It is a variable which belongs to the class and not toobject(instance)
  • Static variables are initialized only once , at the start of the execution . These variables will be initialized first, before the initialization of any instance variables
  • A single copy to be shared by all instances of the class
  • A static variable can be accessed directly by the class name and doesn’t need any object
  • Syntax : <class-name>.<variable-name>

static method

  • It is a method which belongs to the class and not to the object(instance)
  • A static method can access only static data. It can not access non-static data (instance variables)
  • A static method can call only other static methods and can not call a non-static method from it.
  • A static method can be accessed directly by the class name and doesn’t need any object
  • Syntax : <class-name>.<method-name>
  • A static method cannot refer to “this” or “super” keywords in anyway

Side Note:

  • main method is static , since it must be be accessible for an application to run , before any instantiation takes place.

Lets learn the nuances of the static keywords by doing some excercises!

Assignment: To Learn working of static variables & methods

Step 1) Copy the following code into a editor

public class Demo{
 public static void main(String args[]){
 Student s1 = new Student();
 Student s2 = new Student();

class Student {
int a; //initialized to zero
static int b; //initialized to zero only when class is loaded not for each object created.

 //Constructor incrementing static variable b

 public void showData(){
 System.out.println("Value of a = "+a);
 System.out.println("Value of b = "+b);
//public static void increment(){


This code is editable. Click Run to Compile + Execute

Step 2) Save & Compile the code. Run the code as, java Demo.

Step 3) Expected output show below


Following diagram shows , how reference variables & objects are created and static variables are accessed by the different instances.


Step 4) It is possible to access a static variable from outside the class using the syntaxClassName.Variable_Name. Uncomment line # 7 & 8 . Save , Compile & Run . Observe the output.

Step 5) Uncomment line 25,26 & 27 . Save , Compile & Run.

Step 6) Error = ? This is because it is not possible to access instance variable “a” from static method “increment“.

static block

The static block, is a block of statement inside a Java class that will be executed when a class is first loaded in to the JVM

class Test{
  //code goes here

A static block helps to initialize the static data members, just like constructors help to initialize instance members

Why Johnny Can’t Write Multithreaded Programs

Programming for multiple threads is not fundamentally different from writing an event-oriented GUI application or even a straight up sequential application. The important lessons of encapsulation, separation of concerns, loose coupling, etc. all apply. But developers get into trouble with multiple threads when they don’t apply those lessons; instead they try to apply the mostly-irrelevant bits of information they learned about threads and synchronization primitives from introductory multithreading texts.

Some people, when confronted with a problem, think, “I know, I’ll use regular expressions.” Now they have two problems. –Jaimie Zawinski

Some people, when confronted with a problem, think, “I know, I’ll use threads!” Now they have 10 problems. –Bill Schindler

Too many programmers writing multithreaded programs are like Mickey Mouse inThe Sorcerer’s Apprentice. They learn to create a bunch of threads and get them mostly working, but then the threads go completely out of control and the programmer doesn’t know what to do.

Unlike Mickey, those programmers don’t have the luxury of a kindly master wizard who can wave his magic wand and restore sanity. Instead, the programmer resorts to all manner of ugly hacks in an attempt to fix problems as they pop up. The result is invariably an overly complicated, restrictive, fragile, and unreliable application that’s prone to deadlocks and other multithreading hazards. Not to mention unexplained crashes, poor performance, and incomplete or incorrect results.

You’ve probably wondered why that is. Perhaps you’ve accepted the common fallacy that “Multithreading is hard.” It’s not. If a multithreaded program is unreliable it’s most likely due to the same reasons that single-threaded programs fail: The programmer didn’t follow basic, well known development practices. Multithreaded programs seem harder or more complex to write because two or more concurrent threads working incorrectly make a much bigger mess a whole lot faster than a single thread can.

The “multithreading is hard” fallacy is propagated by programmers who, out of their depth in a single-threaded world, jump feet first into multithreading – and drown. Rather than re-examine their development practices or preconceived notions, they stubbornly try to “fix” things, and use the “multithreading is hard” excuse to justify their unreliable programs and missed delivery dates.

Note that I’m talking here about the majority of programs that use multithreading. There are difficult multithreading scenarios, just as there are difficult scenarios in the single-threaded world. But those are relatively rare. For the majority of what most programmers do, the problems just aren’t that complicated. We move data around, transform it, perhaps do some calculations from time to time, and finally store the results in a database or display them on the screen.

Upgrading a typical single-threaded program so that it uses multiple threads isn’t (or shouldn’t be) very difficult. It becomes difficult for two reasons:

  • Developers fail to apply simple, well known development practices; and
  • Most of what they were taught in introductory multithreading materials is technically correct but completely irrelevant to the problems at hand.

The most important concepts in programming are universal; they apply equally to single-threaded and multithreaded programs. Programmers who drown in a sea of threads haven’t learned the important lessons from writing single-threaded programs. I know this because they make the same fundamental mistakes in their multithreaded programs as they do in their single-threaded programs.

Probably the most important lesson to be learned from the past 60 years of software development is that global mutable state is bad. Really bad. Programs that depend on global mutable state are harder to reason about and generally less reliable, because there are too many possible ways for the state to change. There is a huge amount of research to back up that generalization, and countless design patterns whose primary purpose is to implement some type of data hiding. The best thing you can do to make your programs easier to reason about is to eliminate as much global mutable state as possible.

In a single-threaded sequential program, the likelihood of data being mangled is proportional to the number of components that can modify that data.

It’s usually not possible to completely eliminate global state, but we developers have very effective tools for strictly controlling which parts of a program can modify it. In addition, we’ve learned to create restrictive API layers around primitive data structures so that we also control how those data structures are changed.

The problems of global mutable state became more apparent in the late ’80s and early ’90s with the widespread use of event-oriented programming. Programs no longer start at the beginning and follow a single predictable path to conclusion. Instead, the program has an initial state and events occur at unpredictable times in an unpredictable order. The code is still single-threaded, but it’s asynchronous. The likelihood of data being mangled increases because the order in which events can occur is a factor. It’s not uncommon to find that if event A occurs before event B, then everything’s fine. But if A follows B, especially if event C occurs in between, then the data is mangled beyond recognition.

Adding concurrent threads complicates the problem even further because multiple methods can manipulate the global state at the same time. It becomes impossible to reason about how the global state is changing. Not only is the order of events unpredictable, but multiple threads of execution can be updating the state at the same time. At least in the asynchronous case you can guarantee that one event will complete its processing before any other event can start. In short, it is possible to say with certainty what the global state will be at the end of an event’s processing. With multiple threads it’s impossible in the general case to say which events will execute concurrently, and it’s therefore impossible to say what the global state is at any given point in time.

A multithreaded program with extensive global mutable state is one of the best demonstrations of the Heisenberg uncertainty principle I know of. It’s impossible to examine the state without changing the program’s behavior.

When I launch into my prepared rant about global mutable state (a somewhat expanded version of the last few paragraphs), programmers roll their eyes and tell me that they already know that. If they do know that, their programs don’t show it. The programs are filled with global mutable state, and the programmers wonder why their programs don’t work.

Not surprisingly, the most important part of creating a multithreaded program is design: figuring out what the program has to do, designing independent modules to perform those functions, clearly identifying what data each module needs, and defining the communications paths between modules. [Also: designing the project team’s t-shirt. Some things take priority. –Ed.] The fundamental process is no different from designing a single-threaded program. The key to success is, as with a single-threaded program, limiting interactions between the modules. If you eliminate shared mutable state, then data sharing problems are impossible.

You might think that you can’t afford the time to design your application so that it doesn’t use global state. In my opinion you can’t afford not to. Trying to manage global mutable state kills more multithreaded programs than anything else. The more you have to manage, the more likely it is that your program will crash and burn.

Most real world programs require some shared state that can be changed, and that’s where programmers most often get into trouble. Seeing the need for sharing state, programmers often reach into their multithreading toolbox and pull out the only tool they have: the all-purpose lock (critical section, mutex, or whatever it’s called in their particular language). They figure, I suppose, that they can eliminate the data sharing problems with mutual exclusion.

The number of problems you can encounter with a single lock is astounding. There are race conditions to think about, gating problems with an overly broad lock, and fairness issues, just to name a few. If you have multiple locks, especially nested locks, you have to worry about deadlock, livelock, lock convoys, and other concurrency hazards in addition to the problems associated with a single lock. Things get complicated in a hurry.

When writing or reviewing application code, I have a simple rule of thumb that rarely fails: If you used a lock, you probably did something wrong.

That statement can be taken two ways:

  1. If you need a lock, then you probably have global mutable state that has to be protected against concurrent updates. The existence of global mutable state indicates a flaw in the application’s design, which you should review and change.
  2. Locks are difficult to use correctly, and locking bugs can be incredibly difficult to isolate. The likelihood of there being an error in the way you used the lock is very high. If I see a lock, especially in a program that exhibits unusual behavior, the first place I look for the failure is the code that depends on the lock being used correctly. And that’s where I usually find it.

Both of those interpretations apply.

Multithreading isn’t hard. Properly using synchronization primitives, though, is really, really, hard. You probably aren’t qualified to use even a single lock properly. Locks and other synchronization primitives are systems level constructs. People who know a lot more about multithreading use those constructs to build concurrent data structures and higher level synchronization constructs that mere application programmers like you and I use in our programs. Application programmers should use the low-level synchronization primitives about as often as they make direct device driver calls: almost never.

Trying to solve a data sharing problem with locks is like trying to put out a fire by throwing liquid oxygen on it. As with fires, prevention is the best solution. If you eliminate shared state, you have no reason to misuse those synchronization primitives.

Most of what you know about multithreading is irrelevant

Introductory multithreading materials explain what threads are. Then they launch into discussions of how to make those threads work together in various ways, such as controlling access to shared data with locks and semaphores, and perhaps controlling when things happen with events. There’s detailed discussion of condition variables, memory barriers, critical sections, mutexes, volatile fields, and atomic operations. You’re given examples of how to use those low level constructs to do all manner of systems level things. By the time a programmer is halfway through that material, she thinks she knows how to use those primitives in her applications. After all, if you understand how to use something at the systems level, using it at the application level should be trivial, right?

This is like teaching a teenager how to build an internal combustion engine from discrete parts and then, without the benefit of any driving instruction, setting him behind the wheel of a car and turning him loose on the roads. The teenager understands how the car works internally, but he has no idea how to drive it from point A to point B.

Knowing how threads work at the systems level is mostly irrelevant to understanding how to use them in an application program. I’m not saying that programmers shouldn’t know how things work under the hood, just that they shouldn’t expect that knowledge to be directly applicable to the design or implementation of a business application. After all, knowing the details of the intake, compression, combustion, and exhaust cycle doesn’t help you in getting from home to the grocery store and back.

Introductory multithreading textbooks (and computer science courses) shouldn’t be teaching those low level constructs. Rather, they should concentrate on common classes of problems and show developers how to use higher level constructs to solve those problems. For example, a large number of business applications are in concept extremely simple programs: They read data from one or more input devices, apply some arbitrarily complex processing to that data (perhaps querying some other stored data in the process), and then output the results.

These programs very often fit nicely into a producer-consumer model with three threads:

  • The input thread reads data and places it on the input queue.
  • The processing thread reads records from the input queue, processes them, and puts them on the output queue.
  • The output thread reads records from the output queue and stores them.

The three threads operate independently and communicate through the queues. Although technically those queues are shared state, in practice they are communications channels with their own internal, synchronization. The queues support multiple producers and consumers, all adding or removing items concurrently.

Because the input, processing, and output are each isolated, it’s easy to change their implementations without affecting the rest of the program. As long as the queue data types remain unchanged, the individual pieces can be refactored at will. In addition, because the queues handle an arbitrary number of producers and consumers, adding more producers or consumers is no problem. There could be a dozen input threads all writing to the same queue, or multiple processing threads removing input items and crunching the data. Within the confines of a single computer, this model scales well.

Perhaps most importantly, modern programming languages and libraries make it easy to create a producer-consumer application. In .NET you have concurrent collections and TPL Dataflow. Java has the Executer service, BlockingQueue, and other classes in the java.util.concurrent namespace. In C++ you have the Boost threading library and Intel’s Thread Building Blocks. Microsoft introduced its Asynchronous Agents with Visual Studio 2013. Similar libraries are available for Python, Javascript, Ruby, PHP, and for all I know many other languages. You can create a producer-consumer application with any of those packages without ever having to use a lock, semaphore, condition variable, or any other synchronization primitive.

Granted, those libraries likely make liberal use of many different synchronization primitives. That’s okay. Those libraries were written by people who know multithreading a whole lot better than does your average application programmer. Using a library like that is no different from using a language’s runtime library, or writing in a high level language rather than Assembly language.

The producer-consumer model is just one example. The libraries I mentioned above include classes with which you can implement many common multithreading design patterns without once dipping into low-level multithreading. It’s possible to create extensive multithreading applications without knowing a thing about how threads and synchronization work under the hood.

Use the libraries

Writing programs that use multiple threads is not fundamentally different from writing single-threaded synchronous programs. The important lessons of encapsulation and data hiding are universal, and become even more important when multiple concurrent threads are involved. If you ignore those important lessons, then no amount of low level threading knowledge can save you.

Programmers today have plenty to worry about at the application level without having to think about systems-level things. As applications become more involved, we increasingly hide complexity behind API layers. We’ve been doing this for decades. One could make a good argument that hiding complexity from programmers is the primary reason they are able to create complex applications. After all, don’t we already hide the complexities of the file system, the UI message loop, low-level communication protocols, etc.?

Multithreading concepts should be no different. The majority of multithreading scenarios business programmers are likely encounter are well known and implemented in libraries that hide the bewildering complexity of dealing with concurrency. We should use those libraries in the same way that we use libraries of user interface controls, communications protocols, and the countless other tools that simplify our jobs. Leave low level multithreading to the people who know what they’re doing: the ones who write the libraries we use to build real programs.

Jim Mischel is a developer with Professional Datasolutions, Inc., a leading provider of software, hardware, and professional services to convenience retailers and wholesale petroleum marketers. When he’s not banging out code or writing about his experiences, he’s probably putting in miles on his bike or working on his latest wood carving project. Keep up with Jim on his blog.

Thread Interference

Consider a simple class called Counter

class Counter {
    private int c = 0;

    public void increment() {

    public void decrement() {

    public int value() {
        return c;


Counter is designed so that each invocation of increment will add 1 to c, and each invocation of decrement will subtract 1 from c. However, if a Counter object is referenced from multiple threads, interference between threads may prevent this from happening as expected.

Interference happens when two operations, running in different threads, but acting on the same data, interleave. This means that the two operations consist of multiple steps, and the sequences of steps overlap.

It might not seem possible for operations on instances of Counter to interleave, since both operations on c are single, simple statements. However, even simple statements can translate to multiple steps by the virtual machine. We won’t examine the specific steps the virtual machine takes — it is enough to know that the single expression c++ can be decomposed into three steps:

  1. Retrieve the current value of c.
  2. Increment the retrieved value by 1.
  3. Store the incremented value back in c.

The expression c-- can be decomposed the same way, except that the second step decrements instead of increments.

Suppose Thread A invokes increment at about the same time Thread B invokes decrement. If the initial value of c is 0, their interleaved actions might follow this sequence:

  1. Thread A: Retrieve c.
  2. Thread B: Retrieve c.
  3. Thread A: Increment retrieved value; result is 1.
  4. Thread B: Decrement retrieved value; result is -1.
  5. Thread A: Store result in c; c is now 1.
  6. Thread B: Store result in c; c is now -1.

Thread A’s result is lost, overwritten by Thread B. This particular interleaving is only one possibility. Under different circumstances it might be Thread B’s result that gets lost, or there could be no error at all. Because they are unpredictable, thread interference bugs can be difficult to detect and fix.