Redefining Go Functions | Hacker News Reader

Redefining Go Functions

pboyd.io

104 pointstodsacerdoti29 comments

This is all possible and quite neat to dive into the specifics, but if you really want to be able swap a std lib call, just turn it into a variable and change it.

  // code.go
  var now = time.Now

  // code_test.go
  func TestCode(t *testing.T) {
      nowSwap := now
      t.Cleanup (func() {
          now = nowSwap
      }
      now = func() time.Time {
          return time.Date(...)
      }
  }

Examples Code: https://github.com/open-telemetry/opentelemetry-go/blob/main... Test: https://github.com/open-telemetry/opentelemetry-go/blob/490f...

The point of the OP is that it changes calls to `time.Now` regardless of whether the code that's calling it uses your variable or not.

I suspect that using a build tag (say `test`) and two function definitions (one that directly calls `time.Now()` and one test-only one that uses a mutable var) will optimize out to zero cost in the non-test case - last I fiddled with that, it was pretty good at consistently inlining trivial wrapper funcs like that.

The compiler will only use _test.go files in the test build - so not an explicit build tag, but a built in one.

That doesn't give you a way to exclude conflicting code, unfortunately, so you can't provide an optimal one for non-test code with it.

And stuff like `func SetTime(...)` in a _test.go file only works for tests in that same package, because other packages don't compile that _test.go and won't have that function defined.

I'm not sure that we are on the same page

Are you saying that you want multiple build tagged files each with a different implementation of the function, all in the same package? (eg. windows, linux, arm)

I mean, the example given by the GP is two implementations in the same package, the standard library version is used in the prod file and the test implementation in the test files - the _test.go is the (implicit) build tag

Or do you have something else in mind?

That is a useful pattern, though I was unclear on why `t.Cleanup` and not `defer`. In case others are curious, too:

> Parallel subtestsWith t.Run(..., func(t testing.T) { t.Parallel(); ... }), the parent test function can return (and thus run its defers) before parallel subtests actually finish.*

Short version is this:

If you are going to get into the business of introducing order dependence to test cases through global state (see my other reply on the parent), you will always want the cleanup to work correctly.

1. Using (testing.TB).Cleanup is a good defensive habit to have if you author test helpers, especially if the test helpers (see: (testing.TB).Helper) themselves do something (e.g., resource provisioning) that requires ordered teardown. Using (testing.TB).Cleanup is better than returning a cancellation or cleanup function from them.

2. (testing.TB).Cleanup has stronger guarantees about when it is called, especially when the test case itself crashes. Example: https://go.dev/play/p/a3j6O9RK_OK.

I am certain that I am forgetting another edge case or two here.

Generally nobody should be designing their APIs to be testable through mutable global state. That solves half the problem here.

This is often the path of pain: https://google.github.io/styleguide/go/best-practices#global....

Its unexported for that reason. You only change it in tests.

Just for the record - this is package local - it's fine within the package it is defined in, but no other package will use the implementation, they will all use the standard library.

Others have linked to the much more "fun" https://github.com/bouk/monkey which is an actual monkey patch, in that it changes the code that is called from anywhere in the runtime

Other prior art: https://github.com/bouk/monkey with accompanying blog post https://bou.ke/blog/monkey-patching-in-go/

Wow 11 years ago, takes me back...

I've used a different approach to this: there's no real need to modify the compiled binary code because Go compiles everything from source, so you can patch the functions at the source level instead: https://github.com/YuriyNasretdinov/golang-soft-mocks

The way it works is that at the start of every function it adds an if statement that atomically checks whether or not the function has been intercepted, and if it did, then executes the replacement function instead. This also addresses the inlining issue.

My tool no longer works since it was rewriting GOPATH, and Go since effectively switched to Go Modules, but if you're persistent enough you can make it work with Go modules too — all you need to do is rewrite the Go module cache instead of GOPATH and you're good to go.

This is cool, just don't let Rob Pike see this, he will have a conniption. Glad you called out that it shouldn't be used because this is about the most magical thing I have ever seen in Go

The lesson of the post is that Go is compiled to machine-specific object code mapped to a non-writable text segment in an OS-specific process. What's really magical is that compilers save you from such dangerous and non-portable details.

There's a proper way to virtualize time for tests:

https://pkg.go.dev/testing/synctest#hdr-Time

> There you go. It’s 5PM. It’s always 5PM.

That reminded me of the Go Playground, where it is always 2009

https://go.dev/play/p/VrWYHGbtc6m

Spirit of Perl is still alive

Yikes, I don't see any legitimate use for this, other than hacking for the sake of hacking. Interesting read though.

Hot reloading for development loops is _the_ canonical use case for this.

Well, we are on Hacker News after all...

wdym, now it will be possible to implement Wordpress in Go

You could already do that today, via OS IPC mechanisms, at the expense of higher systems resources, with each plugin being its own process.

Wordpress without monkey patching is like Trump without ICE.

I’ve seen this in large C++ systems to allow for a runtime patch, generally to add a simple debug call at the start of a function.

While I would never consider this approach advisable in any language that doesn't build in support for this sort of thing from the start, the thinner the runtime, the less dangerous it is. Go's runtime is fairly thick, and also, concurrent. The odds of something blowing up are rather too high for me to even dream of putting something like this into production in Go. In C++ it may merely be somewhat crazy rather than completely crazy.

(I suppose Rust is arguably an exception to this; thin runtime, but there's a lot of things the replaced function could do that would still blow Rust up if the rest of the code isn't compiled and correctly optimized to account for whatever the new code does.)

You just put in a few NOP instructions at the start of each function to give yourself room :)

In Java you have instrumentation agents that can do arbitrary code rewrite at runtime, including for code that has already been JITted (causing a deoptimization of it). It's first-class supported, unlike in Go.

It's used extensively to (for example) add debugging or tracing to functions in libraries or to sanitize certain code path. At work, we were able to kill off jndi from Log4J to prevent Log4Shell within hours of the announcement this way while waiting for updates of thousands of dependencies.

Reply on news.ycombinator.com