I'm actually just doing that right now (changing my static website to work with Netlify's Push). Presumably, your base HTML looks something like this:

  <link href="{{ "main.css"|asseturl }}" rel="stylesheet" type="text/css" />

Then, your _headers.j2 file should look like this:

  /*
    Link: <{{ "main.css"|asseturl }}>; rel=preload; as=stylesheet

Which will just be replaced with the correct value, same as in your HTML template.