It might be a bad way to compare frameworks, but it’s a pretty good way to compare the performance of HTTP servers and database drivers.
Personally, I think it’s safe to say that both net/http and Cowboy satisfy the HTTP specification, provide a usable interface to build frameworks on top of, are capable of scaling beyond a single core and have proven to be fairly robust. At that point, performance (latency, throughput, memory usage, …) is the only factor. At least, it’s the only factor from the point of view of people working on these components.
I agree that to a consumer, maybe speed isn’t the only factor. They are very different in another respect: one lets you run Go code, the other lets you run Erlang/Elixir code. Maybe that ought to weigh more on your decision than an X% difference in performance.
Or maybe not. I don’t know what your needs are, and I’m not going to assume that performance isn’t really important to you.
People will take whatever value from these benchmarks as they want. Yes, a lot of that will be misguided. And yes, it’s useful to regurgitate the typical “these benchmarks are flawed” and “it doesn’t represent the real world” (which is a pretty condescending thing to say to strangers, really, what people mean is it doesn’t represent THEIR real world).
I know that we’re largely saying the same thing, I just wanted to get the last word in 
