Badgerr's Stuff

Software Development and other (un)interesting things

Golang Away: TCP Chat Server

So I’ve started learning about Golang, skipping all the boring stuff and getting right into the fun, learning the required “boring stuff” as I go. I found some sample code to use as a starting point, but I ended up having to adapt it to fit both my needs, and the apparent evolution in the Go syntax and libraries. It’s far from perfect but I made it work and learned a lot in the process. Let’s get straight into the code, and I’ll explain various Go concepts as we encounter them…

First thing’s first – our package. I’m keeping things simple and using main:

package main

A package represents a collection of functions, sort of like a C++ library with functions declared within a namespace. This directive specifies which package our .go file is part of. We call this package “main” because it’s the main application, and it has a main() function which we’ll get to later.

Now we need to borrow some functionality from other packages, so we’ll import these next:

import "fmt"
import "net"
import "container/list"
import "bytes"

These are all standard libraries that come with Go. There are a lot more. We’re only using four:

  • fmt, implements formatted I/O with functions similar to C’s printf functions;
  • net, implements networking using TCP, UDP, or your own IP protocol – we’ll be using TCP;
  • container/list, implements a doubly-linked list;
  • bytes, implements functions for byte manipulation.

That’s all our dependencies taken care of. Now we define a struct to represent a client:

type Client struct {
	Name string
	Incoming chan string
	Outgoing chan string
	Conn net.Conn
	Quit chan bool
	ClientList *list.List
}

Our client is represented primarily by Name and Conn. It also holds a pointer to a list containing all the connected clients, including itself. The Incoming, Outgoing, and Quit members have an extra keyword in their definition: chan. This means that they represent channels that can transmit data between goroutines.

Woah there, wait a minute. Channels? Goroutines? Let’s step back and look at this.

Without delving into the entirety of Go Concurrency, I’ll try to get the idea across as simply as possible. A Goroutine is a function that runs in parallel with other functions. It silently manages its own threading (but it is NOT a thread – many Goroutines may be running concurrently on a single CPU thread), and silently returns when it has finished. Any function can be called as a Goroutine by using the go keyword before the function call. The caller does not wait for the function to return, so in order to know when a Goroutine has finished, we need Channels.

A channel is like a pipe, where water (our data) flows from one end (function A) to the other (function B). Most of the time, the water isn’t turned on – it only flows when we need to transfer it to the other end. In the context of Go we would create a channel, start our Goroutine, then do some other processing while the Goroutine performs its task. Regardless of who finishes first, the data is always available; when the caller is ready for the data, it uses the <- operator to wait for, and receive, the data from the channel. The Goroutine sends data down the channel using the same operator.

Our caller waits for the water to arrive down the pipe, but in the case where the water arrives first, it waits for the tap to be opened before coming out.

Don’t worry – this will be made clearer when it’s used for real. Let’s continue.

Another difference between C++ and Go is the method of defining what C++ programmers might call “member functions”. In Go, we define functions that perform an operation on a struct outside of the definition. The function can still be accessed with the dot notation (myStructVar.memberFunction()), but defining it this way allows us to implement functions for structures that might otherwise be outside of our control. In C++, inheritance would be a solution to this. We’re going to write some helper functions to assist with our task. First, Read:

func (c *Client) Read(buffer []byte) bool {
	bytesRead, error := c.Conn.Read(buffer)
	if error != nil {
		c.Close()
		Log(error)
		return false
	}
	Log("Read ", bytesRead, " bytes")
	return true
}

Yes, I know. This is backwards too. At least, from a C++ programmer’s perspective. Functions are declared with the func keyword, then in this case a pointer to the object this is being called on, in this case a Client object. Then comes the function name, followed by the parameters (with the same “backwards” definition as we saw in our struct). The return value comes last.

The first line of our function looks the most interesting. Just in case you haven’t noticed yet, functions in Go can return multiple values. I’ve done some Lua development too and I must say I do like this feature in a language. The rest of this code should be pretty straightforward apart from the := operator. This is a shortcut for creating and initializing a variable, instead of var myVar = someValue you simply write myVar := someValue. Also note at this point that Log() is a function we haven’t defined yet. We’ll see that function later.

Our next “member function” for Client looks like this:

func (c *Client) Close() {
	c.Quit <- true
	c.Conn.Close()
	c.RemoveMe()
}

Yep, you guessed it. This closes a client connection. This is our first usage of a channel, specifically the Quit channel from the Client structure. What we’re doing here is sending the boolean value true down the pipe to whatever is waiting at the other end. We’ll see what that is soon enough. RemoveMe() is yet another function that we’ll define later on.

func (c *Client) Equal(other *Client) bool {
	if bytes.Equal([]byte(c.Name), []byte(other.Name)) {
		if c.Conn == other.Conn {
			return true
		}
	}
	return false
}

Equal is our helper function for comparing Clients. It compares the names using a function from the bytes package called Equal, which does exactly as you’d expect. Note the way we cast from a string to a byte array when passing the names to this function.

And our last Client member function, RemoveMe():

func (c *Client) RemoveMe() {
	for entry := c.ClientList.Front(); entry != nil; entry = entry.Next() {
		client := entry.Value.(Client)
		if c.Equal(&client) {
			Log("RemoveMe: ", c.Name)
			c.ClientList.Remove(entry)
		}
	}
}

This function looks through the list of connected clients looking for the one we’re working with, then removes it. Simple!

Our log function is simple, too. So simple it needn’t really exist, but does, so the logging can be changed later without modifying any of the code that wants to write messages:

func Log(v ...interface{}) {
	fmt.Println(v...)
}

Here’s another keyword: interface{}. Everything in Go is derived from interface, so in theory, this can be any datatype. Think of it’s behavior similar to that of boost::variant. As in C and C++, the … specifies a varying number of arguments. For now, we’re just passing this straight into Println.

Can we please get to something a bit more interesting now?

Here’s where the parts start to come together. Three functions that we will be calling as Goroutines, IOHandler, ClientReader, and ClientSender. I will go through them and point out parts that were sticking points for me.

IOHandler:

func IOHandler(Incoming <-chan string, clientList *list.List) {
	for {
		Log("IOHandler: Waiting for input")
		input := <-Incoming
		Log("IOHandler: Handling ", input)
		for e := clientList.Front(); e != nil; e = e.Next() {
			client := e.Value.(Client)
			client.Incoming <-input
		}
	}
}

A function parameter reading from a channel? That’s not confusing at all. Oh wait, it is. Anyway, what this function is doing is running in an infinite loop. You see, there are no while‘s or do‘s in Go. Instead, for can be called without parameters to run indefinitely. Each time this loop runs, it waits for new data in the Incoming channel. This channel object is shared between all clients, as their Outgoing channel. This will become clear in ClientReader(). When the data is ready, it’s stored in input, and then sent down each client’s Incoming pipe, ready for whatever is waiting to collect it.

ClientReader:

func ClientReader(client *Client) {
	buffer := make([]byte, 2048)

	for client.Read(buffer) {
		if bytes.Equal(buffer, []byte("/quit")) {
			client.Close()
			break
		}
		Log("ClientReader received ", client.Name, "> ", string(buffer))
		send := client.Name+"> "+string(buffer)
		client.Outgoing <- send
		for i := 0; i < 2048; i++ {
			buffer[i] = 0x00
		}
	}

	client.Outgoing <- client.Name + " has left chat"
	Log("ClientReader stopped for ", client.Name)
}

Oh look, we’ve found another Go function: make. Go has the concept of slices, which are effectively pointers to a subset of a block of data that was created somewhere else. Under the covers, make finds some space in one of these blocks, and initializes it before returning. That’s a pretty weak explaination but I’m trying to keep this simple. I recommend reading the full description.

ClientReader also has an infinite loop. Well, it’s not infinite, it loops until the client read fails (suggesting a disconnect or closure of the connection). Essentially this function reads data from the TCP connection into buffer then passes it down the client’s Outgoing pipe. Remember our IOHandler function? This is what’s waiting at the other end of the pipe. As soon as send our data to Outgoing, IOHandler springs into life and does its job.

When we get outside this for loop, the client has gone, so we log the event.

ClientSender:

func ClientSender(client *Client) {
	for {
		select {
			case buffer := <-client.Incoming:
				Log("ClientSender sending ", string(buffer), " to ", client.Name)
				count := 0
				for i := 0; i < len(buffer); i++ {
					if buffer[i] == 0x00 {
						break
					}
					count++
				}
				Log("Send size: ", count)
				client.Conn.Write([]byte(buffer)[0:count])
			case <-client.Quit:
				Log("Client ", client.Name, " quitting")
				client.Conn.Close()
				break
		}
	}
}

Our final Goroutine, this time we’re waiting on two channels using a select statement. The select statement is similar to switch, except all the cases are communication operations. If multiple cases are hit, Go pseudo-randomly chooses the more appropriate choice. Our two choices are send data, or quit. The Quit case should be obvious, so I will focus on the sending.

First, remember that both of these client Goroutines are running for every one of our clients. Our IOHandler function took data from ClientReader, and passed it out to all the clients. This is when ClientSender wakes up. For each of the clients that IOHandler passed data to, this function wakes up and tries to send it. This is where I discovered something about arrays and slices, and strings.

Go has a function called len. Guess what it does. Gets the length of the string you pass to it? Yes and no. Given that strings are slices, and slices are, internally, pointers to other bits of data, len gets the length of the underlying array that the string references. This is where I had to make some changes to the code.

First of all, I couldn’t find a strlen type function, so I had to do it myself – you will see my for loop that breaks at the first null terminator. count then contains the length of the string.

Now, when sending the data, we don’t want to send the whole buffer – that would be wasteful. Instead, we send count bytes – this is where slices come in handy. When you have a byte array, you can “slice” it using the array subscript notation, specifying a start and an end point. We use [0:count] in the code above.

Almost there. One more helper function, then main()

ClientHandler is a function to help us accept new connections and register them in the client list:

func ClientHandler(conn net.Conn, ch chan string, clientList *list.List) {
	buffer := make([]byte, 1024)
	bytesRead, error := conn.Read(buffer)
	if error != nil {
		Log("Client connection error: ", error)
	}

	name := string(buffer[0:bytesRead])
	newClient := &Client{name, make(chan string), ch, conn, make(chan bool), clientList}

	go ClientSender(newClient)
	go ClientReader(newClient)
	clientList.PushBack(*newClient)
	ch <-string(name + " has joined the chat")
}

We’re using a slice to create the name string, because otherwise our name is always 1024 bytes long, even if most of it is 0×00. This doesn’t work well for concatenating strings – Go doesn’t really have a concept of a string, it’s just a byte array. It doesn’t make special provisions for null terminators.

Once we have our name we pass all the information to what I can only describe as the Client “constructor”. We pass the name, make a new channel for Incoming, use the channel we got from the function call as Outgoing (this is the same channel that gets passed to IOHandler), and make another channel for Quit. Then we start our Client Goroutines with the go keyword, and add this client to the list.

And now, main. Given the amount of stuff we’ve done so far, you wouldn’t expect main to be that big. And you’d be right.

func main() {
	Log("Hello Server!")

	clientList := list.New()
	in := make(chan string)
	go IOHandler(in, clientList)

	service := ":9988"
	tcpAddr, error := net.ResolveTCPAddr("tcp", service)
	if error != nil {
		Log("Error: Could not resolve address")
	} else {
		netListen, error := net.Listen(tcpAddr.Network(), tcpAddr.String())
		if error != nil {
			Log(error)
		} else {
			defer netListen.Close()

			for {
				Log("Waiting for clients")
				connection, error := netListen.Accept()
				if error != nil {
					Log("Client error: ", error)
				} else {
					go ClientHandler(connection, in, clientList)
				}
			}
		}
	}
}

A few things to note here. First, we create a list to store our clients using New(), just because we’re always going to passing this by pointer. Go has Garbage Collection, so we don’t have to worry about deleting it. Then we create a channel and pass it to IOHandler, along with the list. We also added some usage of the net package here to help set up the connection. We use ResolveTCPAddr to find an IP to start the server on. This seems to automagically figure out IPv4 vs IPv6, and provides us with two functions, Network() and String() to pass to net.Listen().

Once all of this is running without a hitch, we make use of another Go feature, defer. This allows us to specify a function to call at the end of this function. In this case, it’s to stop the TCP server. Note that this runs at the end of the function, not at the end of the scope.

The server is now ready to accept clients. When it does, it starts a Goroutine with ClientHandler, then the rest of the Goroutines kick off and everything “just works”.

This was a long post so I’ll leave it at that. Just remember, you can get the full source code for this server, and a rather buggy C# client that demonstrates it working from my Mercurial Repository.

There’s still a lot more learning for me to do, so until next time, have fun with Go.

3 Responses to “Golang Away: TCP Chat Server”

  • Grégori Marsilio says:

    Hello
    sorry for my english, I’m using google translator

    Very good this server, it worked perfectly.
    You have a customer for it? could send me?

    thank you
    Congratulations for the work….

  • Aslak says:

    Thanks for the excellent blogpost. I coded along and got a pretty good grasp on channels along the way :)

  • Rory says:

    Just trying to be helpful for those self studying:
    In case people still read this article, an exercise for the reader would be to change ClientReader to use a slice to represent valid part of buffer.

    What is the advantage of the change?

    What else can you change to use slices?


Leave a Reply

Disclaimer

The thoughts and opinions expressed on this website are mine and/or those of any authors cited. None of the material on this website represents my employer in any way.