Oops, I fell down the embedded NAND/NOR storage rabbit hole! :oh_no:
Trying to find a data structure that maximizes flash longevity and minimizes RAM and code footprint.
edit: forgot to mention: it should also be simple enough to implement in #Forth or some other minimal language. The aim is #collapseOS -style self-hosting, for maximum #permacomputing coolness.
@csepp have you considered littlefs?
@manifoldslug Yup, it's on my list. I can't use it directly, since I'd like to make my system self-hosting (think CollapseOS) and a C compiler is likely out of reach, but I downloaded its source code and I will look into what data structures it uses.
But I'm not 100% sure LittleFS fits my use case, because I've seen some claims that it targets NOR flash, not NAND. Still, there is probably something useful in there.
@csepp yeah, my second thought was that even if you can't make use of the implementation directly, there is probably a lot to be inspired by within such a neat little filesystem.
self-hosting sounds pretty interesting, could you elaborate on how you plan on doing it? I've been turning over an idea for a project in my head for a while, and my best idea is still extending tcc with code generation for a particular embedded platform and writing an elf loader to go along with it.
@manifoldslug The plan is basically to build a small binary seed and bootstrap a friendlier programming system from that. The problem is that the architecture is very weird (Casio Pocket Viewer) so if I want to make the best use of the hardware I have to do some Weird Shit, like actually make use of segmentation instead of ignoring it.
My current approach is to start from #JonesForth and port it to 16 bit DOS and just play around with the language in DOSBox. I hope I can make the use of segmentation and threading less hacky than DOS by taking a holistic approach to the compiler, runtime, and storage.
A lot of the issues I see with storage and memory management seem to come from having to consider the program to be a black box. I hope that designing all the layers to work together can alleviate some of the irritating quirks of the hardware, like segmentation, or write amplification.
The end result I want is something like a small portable and malleable knowledge base and computational playground, something like a Lisp/Smalltalk/Forth machine in your pocket, plus some database features.
@csepp so, do I understand correctly that the binary seed would be what abstracts away the particulars of generating code for the architecture (x86 or SuperH)? would it be called a bootstrapping compiler or interpreter, then?
sounds ambitious, in either case! from your description of storage/memory design, I get the sense that there may be a parallel with execution, with multitasking.
specifically, it has again boggled my mind recently how comfortable it was to express a blocking context with protothreads, how little it took to make use of them, and how useful they were in an embedded context as an alternative to preemptive scheduling. the difference there feels similar, where threads are assumed to cooperate with the scheduler instead of being black boxes that need to be scheduled preemptively. there are a lot of caveats here, of course, but it was still interesting to find a niche where they made more sense than freertos tasks or something.
@manifoldslug Yeah, pretty much. The binary seed just boots a small Forth from "disk" which then builds up the rest of the system. Although I might want to leave room for caching, in case recompiling on every boot turns out to be too slow.
I'm not sure what this type of system is called, it's a Forth so y'know, not really compiled but not really interpreted either.
Yup, protothreads and/or coroutines are something I'm also considering. I also have this devious thought that I should support async/await, but that will have to wait. But having cooperating threads sounds very appealing for hiding latency. 
I looked a bit into how old DOS and Windows did it and having the signal handling functions also implicitly yield seems Good Enough TM.
