Effective Go

Introduction

Go is a new language. Although it borrows ideas from existing languages, it has unusual properties that make effective Go programs different in character from programs written in its relatives. A straightforward translation of a C++ or Java program into Go is unlikely to produce a satisfactory result—Java programs are written in Java, not Go. On the other hand, thinking about the problem from a Go perspective could produce a successful but quite different program. In other words, to write Go well, it’s important to understand its properties and idioms. It’s also important to know the established conventions for programming in Go, such as naming, formatting, program construction, and so on, so that programs you write will be easy for other Go programmers to understand.

This document gives tips for writing clear, idiomatic Go code. It augments the [3] language specification - https://golang.org/doc/go_spec.html, the Tour of Go - https://tour.golang.org/, and How to Write Go Code - https://golang.org/doc/code, all of which you should read first.

# Arrays

Arrays are useful when planning the detailed layout of memory and sometimes can help avoid allocation, but primarily they are a building block for slices, the subject of the next section. To lay the foundation for that topic, here are a few words about arrays.

There are major differences between the ways arrays work in Go and C. In Go,

• Arrays are values. Assigning one array to another copies all the elements.
  In particular, if you pass an array to a function, it will receive a copy of the array, not a pointer to it.
• The size of an array is part of its type. The types [10]int and [20]int are distinct.
• The value property can be useful but also expensive; if you want C-like behavior and efficiency, you can pass a pointer to the array.

1
2
3
4
5
6
7
8
9

func Sum(a *[3]float64) (sum float64) {
    for _, v := range *a {
        sum += v
    }
    return
}

array := [...]float64{7.0, 8.5, 9.1}
x := Sum(&array)  // Note the explicit address-of operator

But even this style isn’t idiomatic Go. Use slices instead.

Slices

Slices wrap arrays to give a more general, powerful, and convenient interface to sequences of data. Except for items with explicit dimension such as transformation matrices, most array programming in Go is done with slices rather than simple arrays.

Slices hold references to an underlying array, and if you assign one slice to another, both refer to the same array. If a function takes a slice argument, changes it makes to the elements of the slice will be visible to the caller, analogous to passing a pointer to the underlying array. A Read function can therefore accept a slice argument rather than a pointer and a count; the length within the slice sets an upper limit of how much data to read. Here is the signature of the Read method of the File type in package os:

1

func (f *File) Read(buf []byte) (n int, err error)

The method returns the number of bytes read and an error value, if any. To read into the first 32 bytes of a larger buffer buf, slice (here used as a verb) the buffer.

1

n, err := f.Read(buf[0:32])

Such slicing is common and efficient. In fact, leaving efficiency aside for the moment, the following snippet would also read the first 32 bytes of the buffer.

1
2
3
4
5
6
7
8
9
10

var n int
var err error
for i := 0; i < 32; i++ {
    nbytes, e := f.Read(buf[i:i+1])  // Read one byte.
    n += nbytes
    if nbytes == 0 || e != nil {
        err = e
        break
    }
}

The length of a slice may be changed as long as it still fits within the limits of the underlying array; just assign it to a slice of itself. The capacity of a slice, accessible by the built-in function cap, reports the maximum length the slice may assume. Here is a function to append data to a slice. If the data exceeds the capacity, the slice is reallocated. The resulting slice is returned. The function uses the fact that len and cap are legal when applied to the nil slice, and return 0.

1
2
3
4
5
6
7
8
9
10
11
12
13

func Append(slice, data []byte) []byte {
    l := len(slice)
    if l + len(data) > cap(slice) {  // reallocate
        // Allocate double what's needed, for future growth.
        newSlice := make([]byte, (l+len(data))*2)
        // The copy function is predeclared and works for any slice type.
        copy(newSlice, slice)
        slice = newSlice
    }
    slice = slice[0:l+len(data)]
    copy(slice[l:], data)
    return slice
}

We must return the slice afterwards because, although Append can modify the elements of slice, the slice itself (the run-time data structure holding the pointer, length, and capacity) is passed by value.

The idea of appending to a slice is so useful it’s captured by the append built-in function. To understand that function’s design, though, we need a little more information, so we’ll return to it later.

A common operation is to append data to the end of a slice. Go provides a built-in append function that’s good for most purposes; it has the signature

1

func append(s []T, x ...T) []T

The append function appends the elements x to the end of the slice s, and grows the slice if a greater capacity is needed.

1
2
3
4

a := make([]int, 1)
// a == []int{0}
a = append(a, 1, 2, 3)
// a == 

Two-dimensional slices

Go’s arrays and slices are one-dimensional. To create the equivalent of a 2D array or slice, it is necessary to define an array-of-arrays or slice-of-slices, like this:

1
2

type Transform [3][3]float64  // A 3x3 array, really an array of arrays.
type LinesOfText [][]byte     // A slice of byte slices.

Because slices are variable-length, it is possible to have each inner slice be a different length. That can be a common situation, as in our LinesOfText example: each line has an independent length.

1
2
3
4
5

text := LinesOfText{
	[]byte("Now is the time"),
	[]byte("for all good gophers"),
	[]byte("to bring some fun to the party."),
}

Sometimes it’s necessary to allocate a 2D slice, a situation that can arise when processing scan lines of pixels, for instance. There are two ways to achieve this. One is to allocate each slice independently; the other is to allocate a single array and point the individual slices into it. Which to use depends on your application. If the slices might grow or shrink, they should be allocated independently to avoid overwriting the next line; if not, it can be more efficient to construct the object with a single allocation. For reference, here are sketches of the two methods. First, a line at a time:

1
2
3
4
5
6

// Allocate the top-level slice.
picture := make([][]uint8, YSize) // One row per unit of y.
// Loop over the rows, allocating the slice for each row.
for i := range picture {
	picture[i] = make([]uint8, XSize)
}

And now as one allocation, sliced into lines:

1
2
3
4
5
6
7
8

// Allocate the top-level slice, the same as before.
picture := make([][]uint8, YSize) // One row per unit of y.
// Allocate one large slice to hold all the pixels.
pixels := make([]uint8, XSize*YSize) // Has type []uint8 even though picture is [][]uint8.
// Loop over the rows, slicing each row from the front of the remaining pixels slice.
for i := range picture {
	picture[i], pixels = pixels[:XSize], pixels[XSize:]
}

References

[1] Effective Go - The Go Programming Language - https://golang.org/doc/effective_go.html

[2] language specification - https://golang.org/doc/go_spec.html

[3] Tour of Go - https://tour.golang.org/

[4] Go Code - https://golang.org/doc/code

pprint

The pprint module provides a capability to “pretty-print” arbitrary Python data structures in a form which can be used as input to the interpreter. If the formatted structures include objects which are not fundamental Python types, the representation may not be loadable. This may be the case if objects such as files, sockets or classes are included, as well as many other objects which are not representable as Python literals.

Ansible Galaxy: Mac App Store CLI (mas)

Installs mas on macOS, and installs macOS apps from the Mac App Store.

Go maps in action

hredis-rb

Ruby extension that wraps hiredis. Both the synchronous connection API and a separate protocol reader are supported. It is primarily intended to speed up parsing multi bulk replies.

Outline: Access to the free and open Internet

Outline makes it easy to create a VPN server, giving anyone access to the free and open internet.

Outline enables anyone to access the free and open internet more safely by running their own VPN. Running your own VPN server through Outline makes accessing the internet safer and establishes a connection that is harder to block.

pdftoimage

PDFToImage is a ruby gem which allows for conversion of a PDF document into images. It uses poppler_utils to first convert the document to PNG and then allows usage of ImageMagick to convert the image into other formats.

The reasoning behind using poppler_utils is due to the fact that ghostscript occasionally has trouble with certain PDF documents which poppler_utils seems to be able to parse without error.

digital_capital

Currency uppercase amount conversion/RMB uppercase conversion (Convert Arabic numerals into Chinese uppercase, RMB uppercase conversion)

Base64

Base64 is the most popular binary-to-text algorithm used to convert data as plain text in order to prevent data corruption during transmission between different storage mediums. In addition, it is often used to embed binary data into text documents such as HTML, CSS, JavaScript, or XML. Given the high popularity of the algorithm, Base64 is well documented, is supported by various programming languages, has spawned several modifications, and, of course, has its own specification.

rainbow

Rainbow is a ruby gem for colorizing printed text on ANSI terminals.

It provides a string presenter object, which adds several methods to your strings for wrapping them in ANSI escape code - Wikipedia - https://en.wikipedia.org/wiki/ANSI_escape_code. These codes when printed in a terminal change text attributes like text color, background color, intensity etc.