[Awesome Go] Use goworker to run background worker

Posted on 2020-09-19 Edited on 2024-09-12 In Programming Language , Golang (Go) , Awesome Go Views: Word count in article: 4.3k Reading time ≈ 4 mins.

goworker

goworker is a Resque-compatible, Go-based background worker that runs 10 to 100,000* times faster than Ruby-based workers. It allows you to push jobs into a queue using an expressive language like Ruby while harnessing the efficiency and concurrency of Go to minimize job latency and cost.

goworker workers can run alongside Ruby Resque clients so that you can keep all but your most resource-intensive jobs in Ruby.

goworker - https://www.goworker.org/

Installation

The classic way to install

Remember to repalce col-goworker-example with your prefer folder.

1
2
3

$ cd col-goworker-example

$ go get -u github.com/benmanns/goworker@master

Usages

package main

import (
	"fmt"
	"github.com/benmanns/goworker"
)

func myFunc(queue string, args ...interface{}) error {
    fmt.Printf("From %s, %v\n", queue, args)
    return nil
}

func init() {
    settings := goworker.WorkerSettings{
        URI:            "redis://localhost:6379/",
        Connections:    100,
        Queues:         []string{"myqueue", "delimited", "queues"},
        UseNumber:      true,
        ExitOnComplete: false,
        Concurrency:    2,
        Namespace:      "resque:",
        Interval:       5.0,
    }
    goworker.SetSettings(settings)
    goworker.Register("MyClass", myFunc)
}

func main() {
    goworker.Enqueue(&goworker.Job{
      Queue: "myqueue",
      Payload: goworker.Payload{
          Class: "MyClass",
          Args: []interface{}{"hi", "there"},
      },
    })

    if err := goworker.Work(); err != nil {
      fmt.Println("Error:", err)
    }
}

goworker worker functions receive the queue they are serving and a slice of interfaces. To use them as parameters to other functions, use Go type assertions to convert them into usable types.

// Expecting (int, string, float64)
func myFunc(queue, args ...interface{}) error {
	idNum, ok := args[0].(json.Number)
	if !ok {
		return errorInvalidParam
	}
	id, err := idNum.Int64()
	if err != nil {
		return errorInvalidParam
	}
	name, ok := args[1].(string)
	if !ok {
		return errorInvalidParam
	}
	weightNum, ok := args[2].(json.Number)
	if !ok {
		return errorInvalidParam
	}
	weight, err := weightNum.Float64()
	if err != nil {
		return errorInvalidParam
	}
	doSomething(id, name, weight)
	return nil
}

See Getting Started - https://github.com/benmanns/goworker#getting-started to learn more.

FAQs

$ go get -u github.com/benmanns/goworker
go get: github.com/youtube/vitess@none updating to
        github.com/youtube/[email protected]: parsing go.mod:
        module declares its path as: vitess.io/vitess
                but was required as: github.com/youtube/vitess

Because github.com/youtube/vitess has moved to vitess.io/vitess, but the version(v0.1.3) on goworker · pkg.go.dev - https://pkg.go.dev/github.com/benmanns/goworker still import github.com/youtube/vitess.

move vitess.io · benmanns/goworker@7dd3e9d · GitHub - https://github.com/benmanns/goworker/commit/7dd3e9da301700bfb7f463f36873cd3ec5217d67

To get the latest version, you can also use branch master such as:

$ go get -u github.com/benmanns/goworker@master
go: downloading github.com/benmanns/goworker v0.1.4-0.20200828204759-d28a4f34a4d1
go: downloading github.com/cihub/seelog v0.0.0-20140730094913-72ae425987bc
go: downloading vitess.io/vitess v3.0.0-rc.3.0.20181212200900-e2c5239f54d1+incompatible
go get: added github.com/benmanns/goworker v0.1.4-0.20200828204759-d28a4f34a4d1
go get: added github.com/cihub/seelog v0.0.0-20170130134532-f561c5e57575
go get: upgraded golang.org/x/net v0.0.0-20210510120150-4163338589ed => v0.0.0-20210525063256-abc453219eb5
go get: upgraded vitess.io/vitess v0.9.0 => v3.0.0-rc.3.0.20181212200900-e2c5239f54d1+incompatible

git - How to point Go module dependency in go.mod to a latest commit in a repo? - Stack Overflow - https://stackoverflow.com/questions/53682247/how-to-point-go-module-dependency-in-go-mod-to-a-latest-commit-in-a-repo