The legal issues surrounding generative AI are something only the court system can decide. The ethical issues, on the other hand, are things that everyone must decide for themselves. Different people with different value systems will come to different conclusions.

In this episode, Demetri and I open the can of worms that is “AI ethics” and walk through a series of incremental steps people should think through when deciding for themselves if they think generative AI is ethical.

Subscribe now

As always, the art for the thumbnail, podcast cover, and banner of Wading Through AI were all hand-drawn by Anna Rettberg in her signature style. We do not currently use generative AI here at Molly Rocket.

The questions addressed in this video are:

• [00:04] “Quick question (for Casey or anyone with a suggestion!): I am neither a novice nor an experienced programmer, and would like to learn enough C++ to write some useful code, but I am struggling to find good material.”

• [02:18] “In the episode ‘In-order Interleaving’, you have a loop that handles multiple elements per loop iteration. When the list of elements isn’t a multiple of your ‘elements per loop’, we need something after the loop to handle the last few remaining elements. My question is: Is there a particular way to write this ‘residual handling’ part? Whenever I’ve had to do this in my own code it always felt a little awkward.”

• [13:25] “Hey Casey, what’s your take on Arm making their own chips now? Sincerely, an Arm engineer.”

• [17:05] “On the topic of growable arenas, concretely, how do you implement them with respect to the ‘layered’ architecture? To grow them, naturally you would need to mmap or VirtualAlloc more memory, but if you’re far removed from the platform layer and aren’t writing a program that is frame-based (for example, a compiler) and thus have no opportune time to go back to the platform layer and request more memory, what are your strategies? I can only come up with round-tripping back to the OS using a platform-layer API like ‘RequestMoreMemory()’, or something like this. For more context, this would be for the kind of program that also cannot place upper-bounds on memory usage (again, like in a compiler), but still does not want to be malloc’ing/free’ing excessively. I’m keen to hear how you would approach this problem?”

• [26:14] “Have you had occasion to use coroutines to write algorithms that must pause/resume? E.g., protocols in networking or software in a hardware simulator.”

Read more

When it comes to AI, people seem to be having very disparate experiences. Some people become enthusiastic, while others become frustrated, even when apparently attempting similar tasks.

In this episode of Wading Through AI, Demetri and I address the root causes of the feeling a lot of people - both pro- and anti-AI - are having that either they must be crazy, or everyone else must be, because their experiences seem to differ so dramatically from what they are hearing from others.

Subscribe now

As always, the art for the thumbnail, podcast cover, and banner of Wading Through AI were all hand-drawn by Anna Rettberg in her signature style. We do not currently use generative AI here at Molly Rocket.

This is the tenth video in Part 5 of the Performance-Aware Programming series. Please see the Table of Contents to quickly navigate through the rest of the course as it is updated, and the code repository for downloadable code listings.

Read more

Technically, it was dated April 1st, so it may all have been a ruse! But April Fools joke or not, the vast majority of replies took it seriously.

While anyone can subjectively agree or disagree with Notch on the value of DLSS, there’s an underlying objective question regarding his use of the phrase “THE SAME HARDWARE”. Many replies took issue with this wording, claiming it was a clear sign Notch did not understand how DLSS worked.

I’m not so sure about that.

I think there is a good case to be made that - given what little NVIDIA tells us - Notch is not objectively wrong about NVIDIA hardware. I didn’t feel like a contorted X post would be the best way to elaborate on this, so I livestreamed my thoughts on Twitch, YouTube, and here on Substack.

The video is included above for those of you that missed the stream.

In this second episode of Wading Through AI, Demetri and I discuss the recent Anthropic blog post about “parallel Claudes” building a C Compiler, and use it as a case study for how to assess AI announcements.

Subscribe now

As always, the art for the thumbnail, podcast cover, and banner of Wading Through AI were all hand-drawn by Anna Rettberg in her signature style. We do not currently use generative AI here at Molly Rocket.

The questions addressed in this video are:

• [0:00:03] “Building on using free lists to manage objects with complex lifetimes, how do you handle cases where an object type can vary in size?”

• [0:02:33] “Can you go into more detail about how to handle IO with queues vs callbacks? A small example on the fancy light-board would be nice.”

• [0:07:14] “With the recent widespread usage of AI to program in the industry and a fierce push from upper management to enforce usage of AI in development tasks, I am finding myself less and less motivated to program, and also depressed. I have mostly ceased my programming activities outside of working hours.

  Do you have any advice on how to deal with the AI wave and also to continue motivated to program as a hobby and professionally in this environment?”

• [0:20:00] “Now that you are switching to Linux in full swing have you ported your codebases or what’s your strategy? I did the opposite thing and fled from Linux because I wanted to start my first big codebase in a more stable environment so it has been the year of the Windows desktop for me which after running a debloat script hasn’t actually been bad, nevertheless seeing Microsoft’s behavior does have me yearning for the penguin, and knowing how painful releasing something for Linux is has me wondering if programming against the windows API and using Wine for most of one’s graphical software is a valid way to proceed.”

• [0:29:50] “How much of the software you currently write is based on your own thoughts and creativity? I’ve found myself relying heavily on reading and using open-source projects where similar problems were already solved. After a while, this started to make me feel like an impostor. Do great programmers like yourself write their own code, or do they adapt and build on existing work?”

• [0:35:38] “This maybe jumping the gun a fair bit. but when you say back of the envelop calculation, does that include the time complexity of whatever algorithm to do correctly?”

• [0:39:45] “Have you ever written anything in ISPC? From my experience it seems much easier to write than rolling the simd myself, and much more portable.”

• [0:42:10] “Do you have any recommendations on how to use ‘pen & paper’ in the software development process? I’ve noticed that drawing a problem helps me visualize the problem easier than jumping straight into coding. I’m working with MIDI and writing a document explaining in my own words how does MIDI encode data helped me in unexpected ways. I’m wondering if I’m missing out on any other ‘pen & paper’ practice. Any useful tips?”

• [0:46:47] “Hello I come with another question. How does one go about implementing smooth window resizing with Dangerous Thread Crews? I have the two threads and do get non-blocking resizing but the UI contents of the window jiggle and stretch a bit when resizing (which gets way worse with vsync) and nothing I try (like adding synchronization and using WM_WINDOWPOSCHANGING to allow only one resize per frame) gets rid of the apparent discrepancy between the window size and the size my renderer targets that causes some ugliness. The only reference I have of a program that has smooth resizing and doesn’t achieve this by updating on each WM_PAINT is your refterm program but I don’t see any glaring differences between the way you rendered stuff there and it seems like there you just called GetClientRect on each update with no need for anything else... so am I just doing something dumb and missing something or is there some computer wizardry I need to invoke to fix this when programming more complex UIs?”

• [0:51:06] “In this talk about pathfinding in Age of Empires 2, around the 15-16 minute mark, he mentions turning off SIMD because they were losing floating point precision. I don’t understand why that was the case for them, can you provide more insight?”

• [1:01:33] “I’ve been trying to test how many simultaneous loads can I get with the repetition tester when I do 1, 2, 3 or 4 moves in a loop on the two machines I have access to. My laptop (which has Intel Skylake chip) reports that the memory bandwidth doubles only when I go from 1 to 2 moves per loop, which is expected and reproduces your results from the course. But when I do the same on my desktop (which has Intel Raptor lake chip), the results are the same. The Raptor lake apparently have two type of cores: P-cores and E-cores, where E-cores also have 2 ports capable of executing loads, while P-cores are supposed to be equipped with 3 ports of that type (at least that’s what I read in Agner Fog manual). To my understanding it means that I should see a bandwidth bump when going from 1 to 2 moves per loop, and when going from 2 to 3 moves per loop. But that doesn’t happen, I see only one bump (from 1 to 2 moves). I guess that there are some nuances with running the tester on this system that I might not be aware of. But one of them - which is clear to me - is that it is the OS who decides on which core should the tester be run on. So I set the affinity of the tester to CPU1 to make sure that it runs on the P-core. Process Explorer confirmed that it runs on CPU1. But I still could not see the improvement when going from 2 to 3 memory reads. Then I repeated the test with all the cores (one at a time), but I saw no difference in the results.

  It is either my test setup that’s completely broken, or some other factor that I can’t see which prevents the bandwidth improvement of a 3 reads loop. I would be grateful if you could give me some pointers here.”

• [1:07:25] “What are your thoughts about the new dynamicdeopt in msvc that lets you run optimized builds that you can debug with full information because they switch the executable on the fly?”

• [1:08:15] “I’d like to ask you about the pass-by-ref vs pass-by-value ‘debate’. Traditional C++ advice is ‘always pass by const& anything bigger than 8 bytes’, but I’ve recently started seeing some people advocate that 16 byte structs should also be pass-by-value. I know that you couldn’t care less about const, and you do seem to pass by value small stuff without worrying too much about it in your own code, so .. is there anything I’m failing to consider here? is this a stupid thing to worry about in the abstract, or is there some general principle that could be useful to keep in mind here?”

Read more

A few weeks ago, I was approached by long time AI-researcher and personal friend Demetri Spanos about producing a discussion series on emerging AI topics.

Since Demetri has been working on AI technology for decades, now that it’s the hot topic, he regularly gets bombarded with questions from friends who want an opinion from the inside. The idea for the series was to see if we could get those answers out to a wider audience.

As most of you already know, I don’t work with AI myself, so I’m really looking forward to these discussions. Demetri is exceptionally good at avoiding hyperbole, and does a fantastic job providing the technical context you need to draw your own conclusions. As you’ll see, he never tries to sell a specifically positive or specifically negative view of AI, which is very refreshing compared to the normal AI discourse.

I hope you enjoy this first discussion as much as I did. If you’re a subscriber, and have questions or topics you’d like us to cover on future episodes, please feel free to leave a suggestion in the comments! I will be checking regularly to see what our readers are most interested in.

Subscribe now

As always, the art for the thumbnail, podcast cover, and banner of Wading Through AI were all hand-drawn by Anna Rettberg in her signature style. We do not currently use generative AI at Molly Rocket.

This is the ninth video in Part 5 of the Performance-Aware Programming series. Please see the Table of Contents to quickly navigate through the rest of the course as it is updated, and the code repository for downloadable code listings.

Read more

This past Wednesday, one of the topics was “the DRAM shortage”. I thought it might be nice to go through the different types of DRAM and talk about where they are made, just to give some context to the global supply situation (or lack thereof). I livestreamed it on Twitch, YouTube, and of course, right here on Substack.

Read more

The questions addressed in this video are:

• [01:02] “Could you suggest a resource to learn how to work with large datasets that don’t fit into VRAM, or even regular RAM?”

• [07:24] “I do not know how this compares to other approaches, as I’ve not really tried to optimize my profilers much as it’s not really my main area of work, but your profiling explanation made me think of a lock-debug profiler I built. It sounds similar to me at least. Do you agree?

  In my solution, each worker thread owns its profiling buffers completely. I keep a thread-local pointer to a block-based buffer that grows in chunks when needed, and all writes are done by the thread with no locks. The only shared step is that when a thread creates its first block, it registers it once by pushing it into a global linked list so the UI can later iterate all threads. To avoid stalling the writers, I double-buffered it. Each thread allocates two buffers and writes into one of them.”

• [09:46] “About memory usage and the program stack, I believed that this stack could more easily fit in the L1 cache. Isn’t there a higher risk of cache misses at that level if the data to use has been allocated elsewhere? Would it even be noticeable in terms of performance?”

• [14:32] “What’s the best way to multithread a software rasterizer? In my experience, scan line interlacing per triangle was horrific.”

• [18:18] “Do you know why in the file processing test on the same machine on linux mmaping the file could be outperforming all other methods on medium and large mapped chunk sizes, and on windows the same test was worse than everyone else?”

• [21:18] “Regarding callbacks, I think I understood the argument about moving things to a queue instead. However, I also can see the benefit of a callback because that’s synchronous. What if the file IO read some data and stored in a buffer, and the callback now is supposed to handle that read data. If instead you move things to a queue, you’d have to keep allocating new buffers to keep working on those asynchronous reads, because you don’t know when the client code will handle those reads. If instead you can very quickly handle this buffer of read data synchronously in the callback, the file IO can reuse the same buffer to store the next chunk.”

• [25:45] “Can you steelman cases for which an arena/bump allocator (are these the same thing?) is not the preferred way to allocate memory (I imagine it is when lifetimes are not apriori known, but perhaps I am missing more subtlety)? In such cases, what is your preferred method of allocation? Are you forced to go back to new/delete?”

• [33:01] “Re: Dead Code Elimination Prevention Macros, I’ve got identical results with `asm volatile (”“ : “+v”(Value));` which tells the compiler that `Value` is both input and output, forcing it to initialize it, as well as preventing from assuming a specific value. It would generate `vpxor` for `0.0` and `vmovaps` for `0.5`. The advantage here is that it’s quite generic and doen’t depend on operand size/type.Could you please elaborate on your choice of explicitly using instructions?”

• [35:14] “Concerning callbacks, is there a benefit to use callbacks for print-outs/messaging? We have a part of the program that do some calculations which can take time, and it uses callbacks to notify the user about the progress, and waiting to the end of all the computations is not good enough. We also use a callback for a type of mesh calculation that depends on things that this other part of the program shouldn’t know about. (These things were not my decision, but I guess making the module as isolated as possible makes it easier to use it as a module in another program)”

• [37:42] “How do you determine if a solution to a problem is more complex than it needs to be? And for inherently complex and interconnected problems, how do you determine if it needs to be subdivided or not? Is there a general approach for working on a complex system with lots of moving parts? (other than me complaining about it :) )”

Read more

To elaborate, in the official RCA, you will see a lengthy explanation of the state that triggers the bug, as well as the cascading series of failures that took down much of AWS. However, the bug itself is described only by the following single sentence:

Additionally, because the active plan was deleted, the system was left in an inconsistent state that prevented subsequent plan updates from being applied by any DNS Enactors.

That’s not much to go on.

Earlier this month, my hopes for a more complete explanation were bolstered by the sudden appearance of an AWS re:Invent presentation devoted entirely to this particular outage. Sadly, in the hour-long video, only twelve seconds were spent on the actual bug:

Both the RCA and the presentation do a good job of explaining the conditions that triggered the bug. Assuming I understood correctly, the setup goes like this:

1. DynamoDB uses multiple DNS “enactors”, running concurrently, to update DNS records for load balancing (among other things).

2. Enactors use locking and backoff to partially serialize their behavior and make them “easier to reason about”1.

3. Pathological lock contention can cause one enactor to be “enacting” a series of DNS records so old that another enactor will assume they couldn’t still be in use, and try to delete them.

4. An “inconsistent state” occurs when one enactor deletes DNS records it thought were sufficiently old while another enactor updates a DNS endpoint to refer to those same records as authoritative.

5. Enactors will stop operating if they attempt to update a DNS endpoint when it does not already refer to a resolvable name (which it won’t once the “inconsistent state” has occurred).

After all that setup, the story leaves me hanging. To quote the twelve seconds of the presentation:

… but when that plan’s missing, our code couldn’t handle the fact that we tried to point the rollback to nothing …

To me, this is the actual bug, so I wanted to hear about it in detail. Why didn’t the code handle that? Or more specifically, why would the code even need to do anything special to “handle” that case at all?

Let me elaborate. Whenever I’m reading something like an RCA, I’m building a model of the system in my head. I’m sure many of you do this as well - it’s how programmers think through programming problems as they are being described.

My mental model of the AWS system at this point doesn’t yet have a bug. If I imagine myself implementing the code they’re describing here, the pseudo-code version is really just three steps:

1 get the current value of dynamodb.us-east-1.api.aws
  ("plan-110.ddb.aws" in the presentation example)
2 set rollback.ddb.aws to the value from step 1
3 set dynamodb.us-east-1.api.aws to the new value
  ("plan-145.ddb.aws" in the presentation example)

When written in the straightforward way, there should be no issue with step 1 receiving an unresolvable name - some kind of “null”, garbage like 0.0.0.0, or what I assume would have happened in the actual outage: the “old” name whose record had been deleted (shown as plan-110.ddb.aws in the example from the re:Invent presentation).

As far as we’ve been told, performing these DNS record modifications does not require the enactor to actually communicate with any target load balancers, so it shouldn’t matter if the target DNS records exist or not. The rollback entrypoint (rollback.ddb.aws) would just be set to the unresolvable name, just as dynamodb.us-east-1.api.aws had an unresolvable name prior to this update. As far as the explanation goes, the enactors shouldn’t have cared whether these names were resolvable or not.

Additionally, in step 3, the endpoint (dynamodb.us-east-1.api.aws) would be updated to a valid name. Everything except debug code that looks at rollback.dds.aws would work properly had the enactor continued running. On the next update, literally everything - even rollback.ddb.aws - would be back to normal. It would then point to plan-145.ddb.aws, which remains valid and will exist for some time before being cleaned up.

So to me, we have not yet hit the root cause! What is it? Something in the way Amazon wrote the enactor code is the actual root cause. I want to know what it is.

To underscore why I think this is important: taking the explanations at face value, DynamoDB could have been taken down in exactly the same way - causing the same catastrophic outage - simply by manually deleting the target DNS record of a DynamoDB endpoint. According to the available explanation, just that action alone would cause all enactors to cease functioning, and the endpoint would remain an unresolvable name forever.

So it’s hard to call any of the “setup” the root cause, because none of it is actually required for the exact same outage. An accidental manual deletion of an endpoint’s target - even just once - would bring the entire system down, and it would be unrecoverable without operator intervention.

Although the rare, emergent appearance of the “inconsistent state” is something that you would want to fix as well, by itself it wouldn’t have caused AWS to go down catastrophically. At worst it sounds like it would have been a momentary outage. If the enactors weren’t halted by the inconsistent state, one of them would shortly enact a new DNS plan, and things would be back to normal.

In closing, I should also mention that my interest in this is more than idle curiosity. A lot of programming paradigms today rely on architectural patterns I consider dangerous for mission-critical systems. As described in the RCA and presentation, this bug sounds like one that wasn’t really caused by a latent race condition as claimed - it was merely exposed by it. Given the disclosures so far, it sounds far more likely that this bug was actually caused by a bad programming practice in a piece of practical code that communicated with Route53 to change DNS entries.

So don’t leave me hanging, DynamoDB! Tell me what the actual code looked like that took down the internet last October. I want to know how something as basic as exchanging a DNS record was written in such a way that it could permanently halt not one but three different concurrent processes on a case as simple as an unresolvable prior record value.

This is the eighth video in Part 5 of the Performance-Aware Programming series. Please see the Table of Contents to quickly navigate through the rest of the course as it is updated, and the code repository for downloadable code listings.

Read more

The questions addressed in this video are:

• [00:04] “What are your top 5 recommendations for must have projects or applications to build at least once in your life to make you a better programmer ?”

• [14:32] “My question is regarding your previous comment on stack below, where you mentioned that we don't want to do a lot with the stack for performance oriented code. Before this course, I had impression that I should prefer stack allocation to heap allocation (maybe just C++) because I read/watched somewhere saying that 1) stack allocation/dealloc is faster (just move the stack pointer) than heap allocation 2) stack object has more obvious lifetime, 3) stack has no fragmentation concern which improves locality 4) you call on stack object directly while heap object involves indirection.

  I wonder whether my understanding above is correct or not. If not, does that mean I can prefer using heap over stack allocation most of the time? Could you please further comment? Thanks!”

• [32:48] “One idea is to avoid dynamic allocation in the critical path by pushing each measurement into a dedicated logging/profiler thread through a channel. The worker thread would record measurements while the computation threads only perform the minimal ‘send’ operation. But atomic operations, queues, and cross-thread communication can also add overhead, but also distort the original program execution … I’d like advice on how to judge whether this dedicated-thread idea is sound, and in general how to think about designing a low-overhead profiler.”

• [44:20] “What is your opinion on using callbacks for say signal handling? They often seem necessary but I may be overusing them, which leads me to believe my overall architecture could be flawed. But generally speaking, what’s your view on callbacks? Love them or hate them? Do you try to avoid them?”

Read more

This is the seventh video in Part 5 of the Performance-Aware Programming series. Please see the Table of Contents to quickly navigate through the rest of the course as it is updated, and the code repository for downloadable code listings.

Read more

The questions addressed in this video are:

• [00:05] “Question regarding the ‘fat struct’ approach: do you ever find yourself thinking about excess memory consumption caused by some entities having unused fields?”

• [04:29] “About fat structs, you said we would initialize the struct to be able to be A or B depending on the situation, if you imagine that fat struct to be of type Result with cases Success or Error, how would you be able to initialize the Error in the case you have a Success as there was no Error, and vice-versa. This type for example exists in F# and it makes sense to initialize Success only or Error only.”

• [07:40] “Do you have any suggestions for an off the shelf tool to measure bandwidth and flops for a machine your run it on?”

• [13:10] “I sense a recurring theme of reliability and predictability – preferring simple control flow to early returns, preferring simple compilers to strict aliasing, preferring large blocks of memory to pointer festivals, etc. You’ve also spoken about improving _robustness_ by preferring zero/dummy values that just flow through code to null pointers, and preferring handles to pointers (from the iCloud example we just saw).

  What do you mean with “robustness”, and what other techniques can I use to make my code more robust in this way?”

• [18:00] “Do you have any thoughts on Apple’s approach to SIMD?”

• [20:10] “Hi Casey, on the recent podcast with Marco you said that if you could choose a single piece of software to be magically redesigned it would definitely be the browser because the software platform it defines is bad. Could you please elaborate on that? What are the main problems with today’s browsers from your perspective?”

• [22:30] “I’m a bit confused when analyzing the bandwidth I get when reading directly from the volume (using ReadFile with the path ‘\.\C:’ with an offset, for example). As far as I know, that’s the correct way to read system files like the $MFT (which I can currently read properly, by the way).

  When reading 20 GB of contiguous data from the beginning of the volume, I get 2.6 GB/s, and it doesn’t matter whether I use the FILE_FLAG_NO_BUFFERING flag or not—the result is the same. I’d expect something closer to a non-cached read (4.9 GB/s), but I’m getting the same throughput as a cold cached read. I’m not sure where this penalty comes from (assuming the read isn’t triggering extra cache operations since it’s non-buffered).

  Any idea what might be going on here? Do you think these read bandwidths make sense?”

• [25:46] “Will we be able to reuse the coefficients we’re currently using for f64 sine for f32 sine or will we need new ones?”

• [28:24] “I have a question about PC hardware components. I’m not clear on things like the motherboard and chipset. Do they play any role in performance? They vary a lot in price even with similar features, so I assume some aspects must affect overall system performance. Could you give a brief overview of these system parts, if possible?

  Or, to rephrase my question: When you’re building a PC, what do you specifically look at besides just the CPU, RAM, disk and GPU in terms of performance? How do you decide what’s suitable for your specific builds?”

• [36:09] “In my code, I ended up having a single loop over the input that directly produced the haversine sum, rather than splitting parsing and math into two loops. But that means if I want to time parsing vs math, I have to put blocks into the loop, which (seemingly) inevitably introduces a lot of overhead.

  Is there a good way to handle this? The best way I can think of is to instead temporarily comment out parts and just time the rest, though while that’s easy to do with the math part, it seems harder to do for the parsing part, since you still have to somehow produce dummy data for the math while making sure this doesn’t lead to any compiler optimizations you wouldn’t otherwise get.”

• [39:28] “Sorry for repeating the question from the last Q&A, but here is: I have just gotten to it, did it and looked up your solution in QA47 to cross reference. There is one thing that we got differently, and I don’t quite understand your reasoning about it:

  You said that shl rbx, 0 should be recognized by the frontend as a nop and not do anything with flags, but would produce rbx.

  1) If the frontend sees it as a nop, why would it RAW the value of rbx, and not just be a pure nop?

  2) I actually thought that it would not be recognized as a nop (I didn’t find anything about this kind of optimization, i presumed it would be somewhere near zero idiom stuff in the manual), and then it seems like shl will have not only a RAW on rbx, but also on all the flags, as it has be ready that the ALU will say that the shift was 0 and the previous value of flags should be preserved (i. e. RAW)

  So the question is, why is it that rbx is a RAW and flags are skipped, and do you know if there is any place in the docs where such a frontend optimization might be mentioned?”

Read more

This is the sixth video in Part 5 of the Performance-Aware Programming series. Please see the Table of Contents to quickly navigate through the rest of the course as it is updated, and the code repository for downloadable code listings.

Read more

The questions addressed in this video are:

• [00:00:09] “How far can we take SIMD use? What if we put a way bigger onus on eliminating branching and developing parallellizable code? The language K, the simdjson library, the Co-dfns compiler, and the new Box2D engine all blast competitors out of the water by thinking of data parallelism. SIMD use is arduous now - should we start designing our languages and APIs around it? What’s the path to broader cross pollination as an industry? It really seems an untapped potential, right?”

• [00:04:01] “Could you give examples of the kind of substrate work you’re hoping more people take seriously? Are you referring to teams like the WSL or Visual Studio performance teams, or something even deeper (or higher-level)? You’ve cited companies rewriting websites for performance and Microsoft’s console output claims - are these the kinds of things you mean? And if you mean something different, how do you think someone can get involved in that kind of work? I’m trying to understand the bigger vision. Is it that everything on the internet runs as fast as the McMaster-Carr Supply Store website?”

• [00:07:45] “When you critique tools like Git, are you expressing a personal frustration, or do you think there’s an objectively better way these tools could work? It’s hard for me to imagine a world where I don’t need to memorize Git or AWS minutiae - are there products today that represent what you think “just working” should look like?”

• [00:13:00] “What do you think about coroutines?”

• [00:15:50] “Will a VOD of the ‘Research Overview for “The Big OOPs”’ livestream be made available later?”

• [00:16:10] “Do you find that this performance aware development you are talking about also improves the quality of the software in general? Is there any connection? Further, once you decide that something needs to have its own test or tests, what techniques(as in test code organization, handling of test data etc.) and tools you find useful in creating those tests?”

• [00:23:18] “I’m currently onboarding at my first ever job. It’s an giant legacy PHP codebase which primarily uses OOP. I don’t think OOP is great. But if you were forced to use classes for everything, what would be the best way to do it?”

• [00:26:40] “It’s interesting how I get much better performance on Zen3 on Linux, than reference haversine even for the replacement case. ”

• [00:30:47] “I’ve been watching nearly all the BSC talks, and they’ve made me realize I might have a wrong understanding of what a type is. What would be the best definition?”

• [00:35:19] “It seems like there has been a push in recent years towards languages with stronger type systems and static analysis (such as Rust’s borrow checker). Do you think that this trend meaningfully improves software quality, and if so what static analysis tools (both existing and hypothetical) do you think would be the most beneficial for a performance-minded programmer?”

• [00:40:35] “I watched a YouTube video about the Montana mini-computer, and I understood how the concept of a function is implemented at the assembly level. I was wondering: what is a virtual function as defined in a high-level language, and how does that translate to a CPU? Along the same lines, I didn’t fully understand the concept of volatile. It seems to be related to the stack—could you explain how a volatile variable is represented at the assembly/CPU level?”

• [01:04:10] “With Intel AMX becoming more mature or widely-known about, do you think it is or will be possible to start doing things typically done on GPUs (texturing, filtering, convolving, etc) on CPUs in the future? Assuming it will be, do you think GPU vendors will finally start opening up a lá the 30 million line problem in a fight to remain competitive with CPU vendors?”

• [01:13:00] “Unlike games, a lot of the value in the Apple world comes from tight integration with all of their ecosystem and design language (ex: iphone widgets, watch now-playing view, siri searchability, sharing to other apps or airdrop, accessibility integration, etc...) But of course Apple is has a lot of OOP style and “declarative frameworks,” which means that I wouldn’t have control over the code that actually runs these features.

  How can I still use a handmade philosophy of writing my own simpler more focused code rather than depending on lots of slow and volatile libraries?”

• [01:15:36] “do you have any advice for gracefully avoiding or recovering when Apple helpfully deletes your stuff in the background (kills your process when you switch apps, makes you redownload files from icloud)”

• [01:19:21] “In your talk at the Better Software Conference you mention the “fat struct” as a good default option for programming in a systems level language. I think I know the gist of what you mean by this, but I am curious if you have a slightly more formal definition for the term and a general explanation for why it’s a good default approach.”

• [01:27:32] “Most of the course so far talk about programs and assembly for actual chips. How do the aspects of concern under performance aware programming change, if at all, if the target is WASM?”

• [01:30:13] “I perfectly understand why writing to al in a loop is slower than writing to rax, but I get different results for al, ax, eax and rax. The loop writing to al and ax goes at nearly 1/4 of the loop writing to rax, but the eax loop goes at 1/2 that speed. Shouldn’t they run the same since writing to eax does not preserve the upper bits?”

• [01:31:01] “If there’s really no ‘rax’ in a cpu at any given point of time, why (how?) debuggers show a single value what you stop them, or linux will only write a single value in the core dump file? Shouldn’t there be some kind of a tree? I just don’t know if I can’t trust this information for debugging... What if it shows a register value from one branch, but the bug was caused by the value from another, won’t I be misled?”

• [01:40:21] “I was trying to reproduce your results from the RAT and register file lecture. I am running them on Alder Lake chip (i7-12700H). My results were quite the opposite to yours the add only loop was either having similar performance or run much faster compared to mov and add one. I found your article that hints that Alder Lake is able to decouple those chained adds.”

Read more

If you're interested in checking out the stream, it is already live at twitch.tv/theprimeagen. It runs all week, and if I understand correctly, it is somehow 24 hours a day as well! If you're interested in catching the stream while I am on, I should be appearing in the 2PM-6PM range, Pacific Time.

Hope to see you all on the chat!

— Casey

One of the things Marco mentions during the interview is the One Billion Row Challenge, a recent popular Java benchmark. It’s actually quite similar to the Haversine distance problem we use as a recurring example in the Performance Aware Programming series: it involves computing a small number of numerical results from a large amount of data stored in a text file.

After the interview was over, Marco floated the idea of doing a collaboration based on the One Billion Row Challenge. He would write some Java code to solve the challenge, and I would give advice on how to improve its performance. We’d do this a few times, and see how far it would get him toward having a competitive solution.

This sounded like a great idea to me, because not only does it mesh well with what we do here on Computer Enhance, but it would also give me a chance to see first-hand the performance issues that arise when working with Java, a language I don’t generally work with.

So we’re doing it! You can start watching it right now, in fact:

And if you like it, there’s a playlist that will have new episodes as they are posted.

Since there’s a lot to talk about with anything performance-related, from time to time I may also do some follow-up videos here where I go further into some of the details of the challenge. I don’t expect I’ll do any actual Java code - I’ll leave that to Marco - but there are plenty of things that come up during our discussions that would make for good stand-alone posts here on Computer Enhance.

So stay tuned for those as well. Until then, happy viewing everyone!

— Casey