Using Huge Pages in Linux

[joto]

I have recently read this article about Huge Pages on Linux, why they can make sense when running PostgreSQL and how to use them. (There is also this somewhat older article, also about the problems of a PostgreSQL without Huge Pages.)

The first article explains really well how Huge Pages work and how to enable them, so I tried it out and did some preliminary benchmarks and it looks like an osm2pgsql import can be a bit faster with Huge Pages turned on and enough reserved for 16GB shared_buffers. The effect is not that big and the benchmarks weren't very thorough, so I am not sure how large the effect really is, but I am quite sure it won't hurt. And I would expect that the most gain would not be on the import side, but when rendering or otherwise processing the data, so that's something that need more looking into.

Long-standing wisdom has been that any higher settings then 2 GB for shared_buffers doesn't help with osm2pgsql. From looking at the articles it might be that this was due to not having Huge Pages enabled. That's certainly something we should explore.

I wont have that much time at the moment to look into this, but wanted to put this out there in case somebody wants to experiment or already has some experience with this.

[pnorman]

Long-standing wisdom has been that any higher settings then 2 GB for shared_buffers doesn't help with osm2pgsql

I don't think this has been true for a long time. On a memory constrained machine there's tradeoffs between it and osm2pgsql cache, but if you have the memory, more than 2GB is good.