Shitlist Driven Development (2016)

Frequently it's better to change behaviour this way than say, holding meetings and presentations.

A typical example is "we want our developer to write more tests" but there are few existing examples of test code to look at and when you write a test it takes forever to run, so you fix this by fixing the underlaying issues (which is bad or non-existing code), rather than making developers attend TDD presentations and just talking about a "test culture" for example.

As always the key is getting the information to the right people at the right time, and making it more difficult to make the wrong choices then the right choices.

We used this technique to great effect at my previous day job, though under the more neutral name of "ratchet".

When deprecating old code, you can shut that door with a whitelist and gradually force consumers to migrate.

I really like how code is managed at Google.

That way minor alterations like changing a string or numeric value don't remove the method from the whitelist, but alterations to the logic of the given method require you to fix the issue while you're in there poking around already to pass CI.

> Make sure that a certain datastore is only read from in a certain context ... Ensure fallbacks for all uses of a secondary data-store ... joins between tables that have no business being joined

I wish linters / typesystems were extensible enough to do this kind of domain-specific checking. There's a new generation of static analysis that's much more focused on architecture or business rules and is less about code-in-the-small concerns like class methods or operator compatibility.

I think a key to doing large, multi-developer projects, is true modular design, with opaque APIs, and each module with its own project identity and lifecycle.

Not a particularly popular stance, as it means a lot more overhead in each project.

It does reduce the need for shitlists, though.

    # Source

    $ cat example_for_hn.rb
    def foo
      bar
    end

    def bar
      baz
    end

    def baz
      (c1, c2) = caller_locations.first(2)
      puts "Parent caller:      '#{c1.label}' in '#{c1.path}'"
      puts "Grandparent caller: '#{c2.label}' in '#{c2.path}'"
    end

    foo

    # Demo

    $ ruby example_for_hn.rb
    Parent caller:      'bar' in 'example_for_hn.rb'
    Grandparent caller: 'foo' in 'example_for_hn.rb'

    module Shitlist
      def shitlist(method_name, whitelist)
        original_method = instance_method(method_name)
        undef_method(method_name)

        define_method(method_name) do |*args, &block|
          call = caller_locations.first
          passes_whitelist = whitelist.any? do |label, file_pattern|
            call.label == label && call.absolute_path.end_with?(file_pattern)
          end

          unless passes_whitelist
            fail "Shitlisted method! Permitted callers: #{whitelist}"
          end

          original_method.bind(self).call(*args, &block)
        end
      end
    end

    class Example
      extend Shitlist

      def not_on_shitlist
        qux
      end

      def baz
        qux
      end

      def qux
        puts 'Only some methods can call me :)'
      end
      shitlist :qux, 'baz' => 'shitlist.rb'
    end

    $ ruby shitlist.rb
    Only some methods can call me :)
    Traceback (most recent call last):
      2: from shitlist.rb:44:in `<main>'
      1: from shitlist.rb:25:in `not_on_shitlist'
    shitlist.rb:13:in `block in shitlist': Shitlisted method! Permitted callers: {"baz"=>"shitlist.rb"} (RuntimeError)

Of course, you might not want this hijacked method with tracing inside something performance critical. You could always configure the implementation to be a no-op in production.

Most of my experience is C#, where calling deprecated code triggers a warning.

Considering that C# warnings will fail in CI, how would someone do a C# shitlist? Would it require some kind of #pragma, that would stick out like a sore thumb in a code review

It was personal because the idea didn't get a lot of attention when I brought it up.

> “shitlists”: a whitelist of deprecated behaviour. Existing deprecated behaviour is OK and whitelisted.

Your team could change the other teams' code to remove the dependencies. This is usually inefficient. You will waste time learning their code. They may drag out the code review process for weeks or months.

Some teams may refuse to accept your changes and use your service deprecation as political capital to demand more headcount. They may even lie to their managers and claim that the dependency deprecation is justification for a rewrite that they want to do.

There is a technical solution that can help with this social problem: AUTOMATICALLY EXPIRING DEPENDENCY APPROVALS. Configure your library to allow only existing systems to use it, and make them all break on a certain date. Then, instead of forcing the other team to move, they have to move or their build breaks. And if they want to delay turndown they must convince you to change your code. Without automatic expiration, they can delay turndown by simply ignoring you.

Some teams may wait for the dependency expiration, lie saying they didn't know about the turndown, and then demand that you delay the turndown and give them more time. You can work around this with a two-phased turndown. First create a new version of the library that allows only existing clients. Give the library a hideous name so code owners will want to remove it. Example: Deprecated_LiBrArY_YoUr_BuIlD_WiLL_BREaK_oN_20200601_LOL_Were_sERIUS_YOLO_exxtensuns_COme_frun_SVP_DaniELs_OnLY. Then set the existing library to expire in a week and email all users. They can easily switch to the new hideously-named library and in the process acknowledge that they know that their build will break at the specified date.

TLDR: Use expiring white lists so you won't get ignored. Rename your deprecated library to something hideous to motivate code owners to migrate away from it.

Never use profanity in your code, nor in your comments. Lets keep things professional.