IdempotencyPlug - Idempotent POST requests 🔁

News Announcing
, ,
1

Just released IdempotencyPlug, a plug based library that makes POST and PATCH requests idempotent using an Idempotency-Key HTTP header. It follows the IETF Idempotency-Key HTTP Header Field specification draft.

The details

I wrote a blog post on the process and motivations: https://danschultzer.com/posts/idempotencyplug-idempotent-post-requests

Some TLDR excerpts:

An idempotent request ensures that a request only affects the resource at most once. In REST all methods are idempotent except POST and PATCH.

Take the endpoint POST /api/payments that initiate a new payment charge at a payment processor. If the client experiences a network interruption during the request, how can the client safely retry the request without creating a new charge?

These are the requirements for idempotent request handling:

  • Require a single Idempotency-Key HTTP header for all POST and PATCH requests
  • Idempotency-Key value MUST be unique for a URI
  • Idempotency-Key value MUST NOT be reused with a different request payload
  • First-time requests MUST be processed normally, and the response cached
  • Duplicate requests MUST return the cached response
  • Concurrent requests MUST return an error
  • We MUST handle unexpected process termination
  • The cached responses SHOULD expire after 24 hours
  • The cache SHOULD be distributed and persisted

Links

Github: https://github.com/danschultzer/idempotency_plug
Hex: https://hex.pm/packages/idempotency_plug
Docs: https://hexdocs.pm/idempotency_plug/README.html

I hope you find it useful, and as always, any contributions are much appreciated! :smiley:

21 Likes
2

Released v0.2.0 last week that makes customization simpler with MFA configuration values instead of behaviors: idempotency_plug/CHANGELOG.md at main · danschultzer/idempotency_plug · GitHub

1 Like
3

I’ve always wondered, when does this actually happen, if there is a network interruption, then the request will fail anyway from both sides, no? or is this a guarantee in systems where repeating same requests multiple times is a thing?

4

You cannot know if the other side was even reached or reached without our knowledge.

1 Like
5

Yup what @LostKobrakai said. Stripe has a great blog post about it: Designing robust and predictable APIs with idempotency

1 Like
6

Okay, then if we were to treat the request as a single transaction that gets committed only if the client receives the response, then we wouldn’t need this? because at the end of the day we work over TCP and the protocol guarantees delivery.

7

Depending what you mean with receive response, you will need an acknowledgement that the response was received, and then what if you don’t get that and it times out? Gets to the Two Generals Problem. At some point the server must commit, and since that happens server-side you will always have a potential state where the commit occurred, but the client don’t have the acknowledgement that it happened.

In these cases you’ll want to retry the request. Works with idempotent requests like GET, DELETE, and PUT. But if you are dealing with a request that’s not idempotent like POST then you need some way to ensure the retry doesn’t create a new resource, but returns the resource that was created the first time.

1 Like