Hacker News Reader

Comment by suvelx | Hacker News Reader

Every example seems to follow this pattern

  client = pymemcache.client.Client(('127.0.0.1', 11211))  #2 create a client

  # save to memcache client, expire in 60 seconds.
  @ring.memcache(client, expire=60)  #3 lru -> memcache
  def get_url(url):
      return requests.get(url).content

How are you supposed to configure the client at 'runtime' instead of 'compile time' (when the code is executed and not when it's imported)?

Careful placement of imports in order to correctly configure something just introduces delicate pain points. It'll work now, but an absent minded import somewhere else later can easily lead to hours of debugging.

View on HN

   @ring.memcache(client, expire=60)   
   def get_url(url):
       return requests.get(url).content

can be written:

    def get_url(url):
        return requests.get(url).content

    get_url = ring.memcache(client, expire=60)(get_url)

Decorators are just syntactic sugar for that pattern.

You are then welcome to instantiate your ring.memcache object and bind it were it pleases you.

I would have provided a different API though:

   cache = pymemcache.client.Client(('127.0.0.1', 11211))

   @cache.lru(expire=60) # wrapper of ring.cache(client)
   def get_url(url):
       return requests.get(url).content

And accepted the alternative:

   cache = pymemcache.client.Client(conf_factory)

   def get_url(url):
       return requests.get(url).content

   get_url = cache.wraps.lru(get_url, expire=60)

It's better to not expect all people to know about the details of decorators just to use your API, and a factory is a nice hook to have anyway: it say where the code for that dynamic configuration should be and code as documentation is the best doc.

Also a patch() context manager would be nice for temporary caching:

   with cache.patch('module.lru', expire=60):
        get_url()

But it's hard to do in a thread safe way to compromised would have to be made.

Although this is true, it is terrible from a developer UX perspective.

Yes, you can "dynamically"-decorate your functions at run-time using whatever global conditionals.

Yes, you can re-decorate the ring decorators.

But you shouldn't have to.

This design is guilty of the cardinal sin of being un-pythonic.

That's what I said. Read again.

You can use a closure to pass in the configuration.

    def configure_memcache(client_ip, port):
        client = pymemcache.client.Client((client_ip, port))
        @ring.memcache(client, expire=60)
        def get_url(url):
            return requests.get(url).content

        return get_url

Then in your code which imports the above library:

    get_url = configure_memcache('127.0.0.1', 11211)
    result = get_url('https://www.google.com')

I'd rather have a sane API.

    def configure_ring():
        if DEBUG:
            return Ring(backend='debug')
        else:
            return Ring(backend='memcache', ...)

    ring = configure_ring()
    
    @ring.cache(expire=60)
    def get_url(...):
        ...

Tons of other libraries out there that implement this exact pattern.

agree

This assumes you define get_url().

This is a good point. asyncio backends now partially take an initializer function because calling await at importing time is a kind of non-sense.

I think it needs to take also a client-configuration or a client initializer. Any advice from your use case?

I have been thinking about setup and teardown for asyncio apps in Python lately.

The async with block is a nice idea but doesn't deal with the reality that often a resource has multiple consumers. For instance, there might be several components of an application that use a database connection -- I really want to make the connection once and tear it down only after all of the clients of that connection have themselves been torn down.

What I'm imagining the answer to be is something a little bit like the Spring Framework but fundamentally centered around asyncio.

I think there's two common situations that a 'compile time' configuration would not support.

- Loading configuration from `main()` e.g. a configuration in via sys.argv and processed by argparse. - Setting configuration within tests. Unless explicitly told otherwise, I'd expect all tests to be performed against an empty cache. Not to mention, there's no guarantee that I'll have access to a server use during tests.

dogpile.cache author here.

The way dogpile does this is that your decorator is configured in terms of a cache region object, which you configure with backend and options at runtime.

https://dogpilecache.sqlalchemy.org/en/latest/usage.html#reg...

I got this general architectural concept from my Java days, observing what EHCache did (that's where the word "region" comes from).

Surely it's just:

   client = pymemcache.client.Client(('127.0.0.1', 11211))
   cache_wrapper = ring.memcache if some_condition else ring.whatever

   @cache_wrapper(...)
   def ...

Reply on news.ycombinator.com