I saw this bit of text on the cubdb repo’s readme ( GitHub - lucaong/cubdb: Elixir embedded key/value database · GitHub ): Snapshots come at no cost: nothing is actually copied or written on disk or in memory, apart from some small internal bookkeeping. In relation to this code: {x, y} = CubDB.wi…

[image] andzdroid: The function is trying to get a snapshot of the whole database, which is sent back by the genserver call. That expects that “the snapshot” is “the data you store in the db”. It probably isn’t. It’s likely just a pointer or reference to the actual data stored elsewhere.

You’re correct. The message {:snapshot, timeout} is copied when being sent. That doesn’t mean the database itself (as in the data your stored in there) is copied when you do a snapshot though – which is what the documentation is trying to say.

Wouldn’t the whole db be copied in the reply? The function is trying to get a snapshot of the whole database, which is sent back by the genserver call.

If it was a pointer or a reference then it wouldn’t really be a snapshot because then you could see other people’s modifications. Otherwise it would have to be a pointer or a reference to a copy. I believe the docs are implying no copying outside of data transfer between processes.

[image] joey_the_snake: If it was a pointer or a reference then it wouldn’t really be a snapshot because then you could see other people’s modifications. Otherwise it would have to be a pointer or a reference to a copy. That depends on the datastructure used and what information makes up the …

Looking at the source, it seems like a tree structure is copied to the caller. And this tree structure is walked when trying to read particular keys.

Yeah, but that tree structure does in turn refer to a data store and that data store (file by default) holds the actual data. So yes, some stuff is copied (the internal bookkeeping mentioned), but it’s not the database nor the values it stores.

Yeah true. I do see the OP’s point though. I wouldn’t necessarily expect the full tree structure, even if it doesn’t have all the data, based on the description.

I’d imagine that in the grand scheme of things the tree is probably tiny compared to what would need to be moved around if values would be affected – especially when the db reaches sizes where you actually start caring for it not being copied.

@LostKobrakai is essentially right. CubDB uses an append-only Btree: changes are appended to the database file, like in a log file. Basically, every new change causes a new Btree root to be appended to the database file. The root points at the old nodes as well as to some new ones containing the da…

Elixir Programming Language Forum

Question about memory copy

Questions & Help Questions

andzdroid January 11, 2024, 9:04pm 12

Thank you for the detailed explanation!