Diary

Just random thoughts.

The content of this page is highly volatile. I don't want to give each day and each thought a permalink. Some of these thoughts may grow to articles, some never won't. Thus, comments and webmentions will never be implemented for this page. But for archived pages, well, they might be.

Archives

2026-09

2026-10-06

Why am I writing this bullshit? Notes for the documentation? Anyway, let it be.

Basically, realloc can be implemented as

void* realloc(void* addr, unsigned old_size, unsigned new_size)
{
    void* new_addr = alloc(new_size);
    memcpy(new_addr, addr, old_size);
    free(addr, old_size);
    return new_addr;
}

The problem is memcpy that should be avoided.

Then realloc splits into optimized grow and shrink. When optimization fails, memcpy is unavoidable.

In PetWay, the allocator does not track the size of allocated blocks. It's a controversial design decision, but adding even 32-bit unsigned to each allocated block is too wasteful. Its width is close to an average size of English word, not even mentioning the width of size_t on 64-bit systems.

So, realloc and free have old_size argument. Really, I haven't seen a situation where the caller was unable to quickly and reliably get old_size. Oh, yes, C strings. But their only purpose is POSIX API. That extreme case can be worked around.

The allocator uses unsigned instead of size_t. Need memory blocks bigger than 4GB? mmap is the best choice then. Do it youself, in other words. Embracing that extreme case and use size_t in the allocator API does not look good.

Finally, realloc and free actually accept a pointer to pointer:

bool realloc(void** addr_ptr, unsigned old_size, unsigned new_size);
void free(void** addr_ptr, unsigned size);

That's just a precaution against reusing invalid address and double free. However, C language is still dangerous because compilers typically see no difference between void* and void**. The only advantage of void** is catching segfault early instead of double free.

Normally, the use of allocator API should be limited by implementation of strings and storage types such as arrays, maps, lists, etc.

Just looked at most awful code of some open source projects. I won't tell which ones, the point is they all try to localize memory allocation. The worst case is a wrapper around vprintf. If they had normal string API they wouldn't have to call malloc/free in a loop.

Zig scares me each time I see something written in it. Wherever I look I see allocator parameter.

Someone posted on lobsters recently how they dragged that allocator throughout the code to make things working. Another guy asked an LLM for them for a better solution. Yuck!

No, allocator must be invisible. But it must be replaceable.

In PetWay each page header starts with a pointer to allocator. Simply because the allocator contains page catalog.

The allocator is selected in alloc function which has the following prototype:

void* alloc(unsigned size, bool clean, void* shmem);

It's shmem argument. Naming is not perfect as usual, but I can't do better.

shmem can be one of predefined constants to select allocator explicitly or it can point to an existing memory block to use the same allocator. The default allocator is thread-local that does not require any synchronization at all.

Time to draw a big picture?

2026-10-04

Phew, page allocator is more complicated than bitmap sub-allocator. More execution paths, more tests to do.

Alloc and grow are ready, shrink, free, and dump to go.

Two important points that make this allocator look efficient:

  1. mremap can re-map to a reserved block of memory. Moreover, the destination may contain committed pages -- they will be freed. This behavor is described on the man page.

  2. What is more important, it's MREMAP_DONTUNMAP flag which makes possible "graceful free", keeping the memory reserved. Without this flag mremap would leave a hole in a big page. That's not good in a multi-threaded environment.

Page allocator reserves so called big pages which are equal to huge pages. Although huge pages are not used directly, I bear them in mind.

♡♡♡

The amount of good humans does not increase. The opposite is always true. It's like entropy.

Yet another -1 last weekend.

That's sad.

I tried to be nice but they didn't get it. I should not have to, wish I let my organism out to bite them, but I did not want to catch bad karma and get stuck in this shithole for next three lives.

2026-10-01

If source code produces binaries that software developers distribute to bring food on their tables, the source code is the means of production for a developer.

Open source is a way to exproptiate those means of production thus bringing software developers down to slaves and the only thing they can sell is labor.

Some may object, take open source, make binaries, sell, make money. It's like clay, sculpt anything you want. Well, clay is not free, strictly speaking. Even water is not free nowadays. Moreover, following one popular license, if you make anything from free clay, those things must be free too. The only way to make money is to sculpt free things from free clay for someone.

It's a perfect slavery!

Of course it's a shitty analogy. Developers including me love taking such analogies out of thin air. Especially from fields in which they have neither comprehension nor skills.

Anyway, it smells so.

Some jurisdictions place the equal sign between source code and literary work. That not quite correct. Literary work is consumed by readers. By humans. Source code is beyond of comprehension of most of them. It's for machines. It can be written in a human-readable way but, heck, only few are able to do that.

Just thoughts. When it came to the end I started to guess.

♡♡♡

Sometimes windows API looks more consistent than unix mess. But I don't know if windows does not behaves the same way.

In unix the memory is allocated with mmap using anonymous mappings, and pages are cleaned as expected. They contain only zeros. But what if a page was reclaimed and then committed back again?

It stays dirty and needs explicit cleaning!

Unless the mapping was destroyed by munmap and then mmaped back again.

But that's troublesome approach if we need to decommit a hole that should stay reserved. Say, we decommit a page in a reserved region. Another thread may call mmap and break everything.

A man page on madvice explains this:

MADV_FREE: The kernel can thus free these pages, but the freeing could be delayed until memory pressure occurs. For each of the pages that has been marked to be freed but has not yet been freed, the free operation will be canceled if the caller writes into the page.

So, this sequence brings the same dirty page back into the addess space:

mprotect(page, 4096, PROT_NONE);
madvise(page, 4096, MADV_FREE);
mprotect(page, 4096, PROT_READ | PROT_WRITE);

Actually, the page stayed in.

A couple of (ir)relevant links, more or less:

And as a note for myself, do not forget about madvise when you'll get to thansparent huge pages.