The Kernel Trilogy, Volume II: Interrupt Return
This article was originally published in 《2024秋冬季开源操作系统训练营第一、二阶段总结-catme0w》.
Let’s Chat
Hello, I’m a veteran from the Qing Dynasty.
I’m catme0w, you may have already seen my name on the leaderboard.
This time last year, I had already been here once. Why would I come again? The answer will be revealed shortly.
Before continuing, you can check out the record I left last time: 2023开源操作系统训练营第二阶段总结报告-CatMe0w.
As with last time, my log won’t have too much technical play-by-play. If you want to read a good story, you’ve come to the right place; if you’re still struggling with rCore and looking for some tips, I won’t let you leave empty-handed either, catch this:
Written First, TL;DR Guide: Advice You Might Need
- Strongly recommend disabling Copilot, Codeium, and other code completion tools - they completely cannot understand kernels and are almost useless for systems programming; the code they produce will waste a tremendous amount of your time.
- Rubber duck debugging might help.
- Before you 100% understand the codebase, you cannot solve the problems. Trust me.
- After fully understanding the codebase, think through what you’re going to build first, otherwise you’ll be doing major rewrites right out of the gate.
- Be prepared for major rewrites due to wrong direction.
- Be careful with #[repr(C)].
- rustsbi-qemu may be very unstable. Encountering supernatural phenomena? Try a different QEMU version; if you’re using Apple Silicon, also try a different machine.
- gdb might be inefficient - insert more trace!() instead.
Untold Stories
If you read my record from participating in last year’s training camp above, you should already know that I stalled at ch5 and then quit. I actually had some regret in my heart, feeling that I wasn’t really as busy as it seemed, and that it was my own laziness that caused the outcome of having to quit.
Even though rCore was never a big deal, I still felt I “couldn’t hold my head up,” and afterward never mentioned this exquisite little operating system to anyone again, as if it had never happened.
Would I write an operating system myself? Probably never, I thought. Just consider it a dream.
The following summer.
By a very, very chance encounter, I learned that a friend of mine had participated in this year’s spring training camp. The biggest difference between us was that from beginning to end, I only held a “passing is victory” attitude, hoping that advancing was winning, while his goal from the start was to be first on the leaderboard.
I felt as if I’d been slapped hard across the face: Can’t do it, only because I don’t want to.
Fortunately, the training camp never prohibits Qing Dynasty veterans (and now there are more and more of them!), so I have the chance to add a perfect ending to rCore.
rustlings II
I’m also a Rust veteran. These 100 questions - can’t I just merge over my patches from last year?
However, starting this year, rustlings added ten algorithm problems, making it 110 questions. The Qing Dynasty veteran encountered its first obstacle!
I understand Rust’s temperament, know what data structures require unsafe, but I haven’t touched algorithm problems in a long time, and never used Rust to write algorithm problems. Upon opening it, the screen full of generics made me shudder.
This would take some time… but still, just a small interlude. Let’s face our fears and move toward rCore.
rCore II
To be honest, I didn’t understand rCore’s tutorial at all last year. I did complete through ch5, but except for ch3, I myself didn’t know what I had written. My code ran in a way that even I couldn’t understand - probably just happened to hit the test cases correctly.
Looking on the bright side, the chance to make amends has come; looking on the dark side, this is equivalent to having no experience at all!
ch3
No difficulties. With memories from the Qing Dynasty period, I still remembered where to add what in ch3. Finished the battle quickly.
ch4
I still believe to this day that ch4 memory management is the most difficult part of all of rCore, at least knowledge-wise! The task itself is actually not that difficult - not much code, and with understanding of memory management you should quickly find the right path; but if you have even the slightest half-understanding of the memory model, you’ll have an unpleasant time with ch4’s tasks.
Well, I admit I was the latter…
ch5
What did ch5 do again? I forgot again. It just has such little presence. The surrounding ch4 and ch6 are both scream-inducing levels, by comparison, ch5 is somewhat hard to leave a deep impression.
Oh I remember now, it’s spawn. Really don’t have much to say. After being beaten up by the previous two chapters, you should be able to handle this chapter quickly.
ch6
I arrived at the place I couldn’t conquer before. The tough nut has come. At this point, rCore truly shows its power - the BOSS enters Phase 2!
ch6 is slightly easier knowledge-wise, but the task difficulty is completely disproportionate to the tutorial text. The task’s difficulty lies in being very convoluted - to modify a certain function, you might need to modify a chain of code in many different places from front to back, not to mention this module outside the kernel is very difficult to debug.
Here, there’s something I must complain about: programming in Rust can sometimes be really frustrating.
People always say “catching errors at compile time is better than debugging afterward,” but only you know how unpleasant it is using things like Arc and Mutex: it’s right there, so intuitive, but it just won’t let you use it.
At 11 AM that day, I had almost completed the ch6 functionality implementation. I thought, once ch6 passes I’ll go eat lunch.
You can guess how this flag will develop.
Me at 11 PM: 😅
ch8
Almost finished! When I first saw the ch8 tutorial, I was secretly delighted: This simple! Just like returning to ch3!
The task requirements were also simple, even detailing every aspect to be implemented.
Started coding, quickly passed the mutex test case, the first semaphore test case also passed quickly. Only the last test case remained, and I could wrap it up immediately!
Then I got stuck on this “last test case” for over ten hours. 🌚
Either the first test wouldn’t pass, or the second one - the two test cases presented completely incomprehensible results. But no matter how I checked my code, inserting logs everywhere I could, I found each step indeed matched expectations, so why were the results just wrong? I didn’t misunderstand the question either, did I?
“I didn’t misunderstand the question”… wait.
Could it really be that I got the question wrong? Could I have implemented an incorrect algorithm?
At my wit’s end, I started directly writing proofs of this algorithm, and quickly discovered that (as I understood it) this algorithm couldn’t detect any deadlocks at all!
The task description of the algorithm was indeed somewhat confusing and misleading. I realized I should seek external materials. Only at this point did I finally learn this was Dijkstra’s banker’s algorithm.
After viewing the precise description of this algorithm, I finally figured out where my problem was: the need matrix only decreases, never increases, and is definitely not starting from 0 and growing as I initially understood.
Finally, all test cases could pass, ch8 ended, but I felt somewhat amused and exasperated, getting stuck for such a stupid reason lacking technical content made me feel a bit like I’d been played.
rCore EnCore
Thank you for still having the patience to read this far. Perhaps you’ve already noticed that I’m number one on this year’s leaderboard.
After Phase 2 classroom was released, I submitted using the patch method, so the patch still contains the actual completion time: September 21st.
After I completed rustlings, that is, the weekend when registration just opened this year, I invested the entire weekend into speed-running rCore.
For no other reason - I just wanted to slap my last year’s self.
Epilogue
The process of doing rCore labs, the first half is just being at a complete loss: you read the documentation for ages, feel like you’ve got it, open the code and find yourself completely in the dark, not knowing what you’re doing at all.
Then you spend a long time finally roughly figuring out the code structure, get ready to start writing, and find you’re still at a complete loss, don’t even know what to write.
After who knows how long, finally you have a rough idea and can start writing. At this point, you’ve arrived at the second half.
You might have written it in not too much time, or maybe very long, but it definitely won’t run.
Next you’ll spend dozens of hours debugging, with gdb or inserting trace!() everywhere.
However, even using gdb doesn’t help much, because errors often appear in places you can’t imagine.
Sometimes you’ll find the test cases can pass, even randomly pass, and sometimes you’ll find QEMU just doesn’t move at all - rustsbi-qemu is not stable.
When you first discover the tests can pass, even if it’s just probabilistic success, you think you’re close to victory, but you quickly discover, or discover much later, that the direction is completely wrong, and you must tear it all down and start over.
This will happen repeatedly.
Finally, you get closer and closer to the final goal, but each time you’re just short, you can’t help but wonder if you’re really approaching the finish line, or have gotten the direction wrong yet again.
In the end, you might not even remember how many times you overthrew everything in between, how many details you changed. You no longer expect to succeed in one go, but instead learn to enjoy the sense of achievement from solving each small problem.
No matter how many times you fail, how many times you tear it down, you’re already so much stronger than that self who was completely in the dark at the beginning. Maybe the direction was wrong, the code torn down and rewritten, but at least you know you’re making progress, even if each step is difficult. Eventually you’ll discover that these moments of repeated struggle are the most valuable part of the entire process.
After the Epilogue
“Would I write an operating system myself?”
Yes.
I recall something that happened when I was very young - I can barely distinguish exactly when it was.
People were discussing the book “30 Days to Make an Operating System.”
At that time I thought, ah, is this even possible? This book must be very thick, very hard to understand…
In my eyes at that time, aside from the applications ultimately presented to users, everything else about computers was a black box made of magical smoke - muggles couldn’t figure it out.
But after all these years, I finally understand: Can’t do it, only because I don’t want to.
We all can.
The training camp is just a beginning. After this, there’s an even broader world waiting to be explored.