One problem seem to be how benchee’s parallel option is used and how the result is reported. I think 1 ips = all parallel task being run. See Parallel Benchmarking · bencheeorg/benchee Wiki · GitHub for more information
For example, if you set benchee parallel to 1 and pgbench clients and thread to 1 the result was that pgbench was only 5% faster. (If I changed the pgbench sql to run in a transaction they had the same throughput)
This means, the time for serialization and type checking and random generation has nothing to do with the speed as in the simple case they are almost the same.
I don’t know how to properly measure this with benchee. A simplistic way would be to multiply the ips by the parallel number. But I don’t know if that is the right approach either. In your case it changes the results from pgbench not being so much faster (60% to 10% faster depending on settings).
Look into how pgbench does connection pooling. Perhaps it opens up a connection and then just holds that and runs the transactions sequentially on that? Ecto will have to check out a connection from the pool, execute the query, and return the connection. A pool is there to make sure the database doesn’t get overloaded in terms of the number of connections, so there is always going to be overhead. Especially if the number of concurrent requests is higher than the number of connections in the database.
