Go GCC vs GC

For awhile now I’ve been playing around with a stack based language written in Go. The specifics of that project aren’t too important here, but I did notice something interesting today as I tried GCC for the first time with the project. First I wanted to make sure it worked so I ran go-5 test and happily all tests passed. Next, I thought I’d see how the benchmark looked. Here I think I was most surprised. Every single benchmark I had written was between 1.5x & 2x slower with GCC compared to GC. The versions I used were

gcc version 5.3.1 20160101 (Debian 5.3.1-5)
go version go1.4.2 gccgo (Debian 5.3.1-5) 5.3.1 20160101 linux/amd64

go version go1.5.2 linux/amd64

go version go1.4.3 linux/amd64

Go GC 1.4 & 1.5 ended up having very similar performance to one another in my tests. The most interesting tests were two that iterated over a linked list. Below is a recreation of the benchmark that was run. There are really two things that can be considered tested here. Interface method invocation & direct pointer method invocation.


package main

import (
	"fmt"
	"testing"
)

type Lister interface {
	Next() Lister
	Name() string
}

type List struct {
	link Lister
	name string
}

func (l List) Name() string {
	return l.name
}

func (l List) Next() Lister {
	return l.link
}

func BenchmarkLinkedList(b *testing.B) {
	var l Lister
	for i := 1; i < 1000; i++ {
		l = List{link: l, name: fmt.Sprintf("test%d", i)}
	}

	first := "test0"
	b.ResetTimer()

	for i := 0; i < b.N; i++ {
		for ll := l; ll != nil; ll = ll.Next() {
			if ll.Name() == first {
				break
			}
		}
	}
}

type List2 struct {
	link *List2
	name string
}

// Test without the interface function calls
func BenchmarkPureLinkedList(b *testing.B) {
	var l *List2
	for i := 1; i < 1000; i++ {
		l = &List2{link: l, name: fmt.Sprintf("test%d", i)}
	}

	first := "test0"
	b.ResetTimer()

	for i := 0; i < b.N; i++ {
		for ll := l; ll != nil; ll = ll.link {
			if ll.name == first {
				break
			}
		}
	}
}
$ go version
go version go1.4.3 linux/amd64
$ go test -bench .
testing: warning: no tests to run
PASS
BenchmarkLinkedList       100000             14888 ns/op
BenchmarkPureLinkedList  1000000              2166 ns/op
$ go version
go version go1.5.2 linux/amd64
$ go test -bench .
testing: warning: no tests to run
PASS
BenchmarkLinkedList-2             100000             12744 ns/op
BenchmarkPureLinkedList-2        1000000              2179 ns/op
$ go version
go version go1.4.2 gccgo (Debian 5.3.1-5) 5.3.1 20160101 linux/amd64
$ go test -bench .
# _/home/harley/tmp/tests_go/test_linked_list
ar: `u' modifier ignored since `D' is the default (see `U')
# testmain
ar: `u' modifier ignored since `D' is the default (see `U')
testing: warning: no tests to run
PASS
BenchmarkLinkedList        50000             25005 ns/op
BenchmarkPureLinkedList   200000             10096 ns/op

As you can see, Go GC 1.4 & 1.5 aren’t too far off one another (with 1.5 being a bit better in the BenchmarkLinkedList test.) However, compare that to GCC. It’s roughly twice as slow in the first test and nearly 5x slower in the second, when compared to Go GC 1.5.

Here’s the output of pprof for GC 1.5.2

(pprof) top10
21.39s of 21.46s total (99.67%)
Dropped 3 nodes (cum < = 0.11s)
      flat  flat%   sum%        cum   cum%
     6.52s 30.38% 30.38%     16.40s 76.42%  main.LinkedList
     5.01s 23.35% 53.73%      5.01s 23.35%  main.(*List).Name
     5.01s 23.35% 77.07%      5.05s 23.53%  main.PureLinkedList
     4.85s 22.60% 99.67%      4.85s 22.60%  main.(*List).Next
         0     0% 99.67%     21.45s   100%  main.main
         0     0% 99.67%     21.45s   100%  runtime.goexit
         0     0% 99.67%     21.45s   100%  runtime.main

And the output of pprof for GCC 5.3.1

(pprof) top10
25.83s of 25.83s total (  100%)
      flat  flat%   sum%        cum   cum%
    25.83s   100%   100%     25.83s   100%  runtime_sigprof
         0     0%   100%     25.83s   100%  
         0     0%   100%         8s 30.97%  __go_strcmp
         0     0%   100%     18.16s 70.31%  main.LinkedList
         0     0%   100%      4.97s 19.24%  main.Name.N9_main.List
         0     0%   100%      4.63s 17.92%  main.Next.N9_main.List
         0     0%   100%      7.67s 29.69%  main.PureLinkedList
         0     0%   100%     25.83s   100%  main.main
         0     0%   100%     25.83s   100%  runtime_main
         0     0%   100%     25.83s   100%  sig_tramp_info

To me, the interesting thing is that, in both implementations, each of List.Name & List.Next take almost as much time (during 30 seconds) as the PureLinkedList function takes (which is the benchmark function using direct struct & pointer usage.)

That code was run with go run main.go and then go tool pprof http://localhost:6060/debug/pprof/profile while main.go was running. Then I inspected the collected profile with top10.

This code is very similar to the benchmark code above, but uses net/http/pprof

package main

import (
	"flag"
	"fmt"
	"log"
	"net/http"
	"os"
	"time"
)

import _ "net/http/pprof"

type Lister interface {
	Next() Lister
	Name() string
}

type List struct {
	link Lister
	name string
}

func (l List) Name() string {
	return l.name
}

func (l List) Next() Lister {
	return l.link
}

func LinkedList(l Lister) {

	first := "test0"

	for i := 0; i < 10000; i++ {
		for ll := l; ll != nil; ll = ll.Next() {
			if ll.Name() == first {
				break
			}
		}
	}
}

type List2 struct {
	link *List2
	name string
}

// Test without the interface function calls
func PureLinkedList(l *List2) {

	first := "test0"

	for i := 0; i < 10000; i++ {
		for ll := l; ll != nil; ll = ll.link {
			if ll.name == first {
				break
			}
		}
	}
}

var cpuprofile = flag.String("cpuprofile", "", "write cpu profile to file")

func main() {
	var l Lister
	for i := 1; i < 10000; i++ {
		l = List{link: l, name: fmt.Sprintf("test%d", i)}
	}

	var ll *List2
	for i := 1; i < 10000; i++ {
		ll = &List2{link: ll, name: fmt.Sprintf("test%d", i)}
	}

	go func() {
		log.Println(http.ListenAndServe("localhost:6060", nil))
	}()

	for {
		LinkedList(l)
		time.Sleep(time.Millisecond * 300)
		PureLinkedList(ll)
		time.Sleep(time.Millisecond * 300)
	}

	os.Exit(1)
}

I suppose the take away here is direct struct access is going to be faster than interface method calls. Not that this is news or even hard to believe, but is a reminder that tight loops that need speed will do better with direct struct access over interface method calls. Though, and hopefully this is obvious, profiling your application should happen before optimizing code.