This is the eleventh video in Part 5 of the Performance-Aware Programming series. Please see the Table of Contents to quickly navigate through the rest of the course as it is updated, and the code repository for downloadable code listings.

Read more

The questions addressed in this video are:

• [00:04] “Quick question (for Casey or anyone with a suggestion!): I am neither a novice nor an experienced programmer, and would like to learn enough C++ to write some useful code, but I am struggling to find good material.”

• [02:18] “In the episode ‘In-order Interleaving’, you have a loop that handles multiple elements per loop iteration. When the list of elements isn’t a multiple of your ‘elements per loop’, we need something after the loop to handle the last few remaining elements. My question is: Is there a particular way to write this ‘residual handling’ part? Whenever I’ve had to do this in my own code it always felt a little awkward.”

• [13:25] “Hey Casey, what’s your take on Arm making their own chips now? Sincerely, an Arm engineer.”

• [17:05] “On the topic of growable arenas, concretely, how do you implement them with respect to the ‘layered’ architecture? To grow them, naturally you would need to mmap or VirtualAlloc more memory, but if you’re far removed from the platform layer and aren’t writing a program that is frame-based (for example, a compiler) and thus have no opportune time to go back to the platform layer and request more memory, what are your strategies? I can only come up with round-tripping back to the OS using a platform-layer API like ‘RequestMoreMemory()’, or something like this. For more context, this would be for the kind of program that also cannot place upper-bounds on memory usage (again, like in a compiler), but still does not want to be malloc’ing/free’ing excessively. I’m keen to hear how you would approach this problem?”

• [26:14] “Have you had occasion to use coroutines to write algorithms that must pause/resume? E.g., protocols in networking or software in a hardware simulator.”

Read more

This is the tenth video in Part 5 of the Performance-Aware Programming series. Please see the Table of Contents to quickly navigate through the rest of the course as it is updated, and the code repository for downloadable code listings.

Read more

The questions addressed in this video are:

• [0:00:03] “Building on using free lists to manage objects with complex lifetimes, how do you handle cases where an object type can vary in size?”

• [0:02:33] “Can you go into more detail about how to handle IO with queues vs callbacks? A small example on the fancy light-board would be nice.”

• [0:07:14] “With the recent widespread usage of AI to program in the industry and a fierce push from upper management to enforce usage of AI in development tasks, I am finding myself less and less motivated to program, and also depressed. I have mostly ceased my programming activities outside of working hours.

  Do you have any advice on how to deal with the AI wave and also to continue motivated to program as a hobby and professionally in this environment?”

• [0:20:00] “Now that you are switching to Linux in full swing have you ported your codebases or what’s your strategy? I did the opposite thing and fled from Linux because I wanted to start my first big codebase in a more stable environment so it has been the year of the Windows desktop for me which after running a debloat script hasn’t actually been bad, nevertheless seeing Microsoft’s behavior does have me yearning for the penguin, and knowing how painful releasing something for Linux is has me wondering if programming against the windows API and using Wine for most of one’s graphical software is a valid way to proceed.”

• [0:29:50] “How much of the software you currently write is based on your own thoughts and creativity? I’ve found myself relying heavily on reading and using open-source projects where similar problems were already solved. After a while, this started to make me feel like an impostor. Do great programmers like yourself write their own code, or do they adapt and build on existing work?”

• [0:35:38] “This maybe jumping the gun a fair bit. but when you say back of the envelop calculation, does that include the time complexity of whatever algorithm to do correctly?”

• [0:39:45] “Have you ever written anything in ISPC? From my experience it seems much easier to write than rolling the simd myself, and much more portable.”

• [0:42:10] “Do you have any recommendations on how to use ‘pen & paper’ in the software development process? I’ve noticed that drawing a problem helps me visualize the problem easier than jumping straight into coding. I’m working with MIDI and writing a document explaining in my own words how does MIDI encode data helped me in unexpected ways. I’m wondering if I’m missing out on any other ‘pen & paper’ practice. Any useful tips?”

• [0:46:47] “Hello I come with another question. How does one go about implementing smooth window resizing with Dangerous Thread Crews? I have the two threads and do get non-blocking resizing but the UI contents of the window jiggle and stretch a bit when resizing (which gets way worse with vsync) and nothing I try (like adding synchronization and using WM_WINDOWPOSCHANGING to allow only one resize per frame) gets rid of the apparent discrepancy between the window size and the size my renderer targets that causes some ugliness. The only reference I have of a program that has smooth resizing and doesn’t achieve this by updating on each WM_PAINT is your refterm program but I don’t see any glaring differences between the way you rendered stuff there and it seems like there you just called GetClientRect on each update with no need for anything else... so am I just doing something dumb and missing something or is there some computer wizardry I need to invoke to fix this when programming more complex UIs?”

• [0:51:06] “In this talk about pathfinding in Age of Empires 2, around the 15-16 minute mark, he mentions turning off SIMD because they were losing floating point precision. I don’t understand why that was the case for them, can you provide more insight?”

• [1:01:33] “I’ve been trying to test how many simultaneous loads can I get with the repetition tester when I do 1, 2, 3 or 4 moves in a loop on the two machines I have access to. My laptop (which has Intel Skylake chip) reports that the memory bandwidth doubles only when I go from 1 to 2 moves per loop, which is expected and reproduces your results from the course. But when I do the same on my desktop (which has Intel Raptor lake chip), the results are the same. The Raptor lake apparently have two type of cores: P-cores and E-cores, where E-cores also have 2 ports capable of executing loads, while P-cores are supposed to be equipped with 3 ports of that type (at least that’s what I read in Agner Fog manual). To my understanding it means that I should see a bandwidth bump when going from 1 to 2 moves per loop, and when going from 2 to 3 moves per loop. But that doesn’t happen, I see only one bump (from 1 to 2 moves). I guess that there are some nuances with running the tester on this system that I might not be aware of. But one of them - which is clear to me - is that it is the OS who decides on which core should the tester be run on. So I set the affinity of the tester to CPU1 to make sure that it runs on the P-core. Process Explorer confirmed that it runs on CPU1. But I still could not see the improvement when going from 2 to 3 memory reads. Then I repeated the test with all the cores (one at a time), but I saw no difference in the results.

  It is either my test setup that’s completely broken, or some other factor that I can’t see which prevents the bandwidth improvement of a 3 reads loop. I would be grateful if you could give me some pointers here.”

• [1:07:25] “What are your thoughts about the new dynamicdeopt in msvc that lets you run optimized builds that you can debug with full information because they switch the executable on the fly?”

• [1:08:15] “I’d like to ask you about the pass-by-ref vs pass-by-value ‘debate’. Traditional C++ advice is ‘always pass by const& anything bigger than 8 bytes’, but I’ve recently started seeing some people advocate that 16 byte structs should also be pass-by-value. I know that you couldn’t care less about const, and you do seem to pass by value small stuff without worrying too much about it in your own code, so .. is there anything I’m failing to consider here? is this a stupid thing to worry about in the abstract, or is there some general principle that could be useful to keep in mind here?”

Read more

This is the ninth video in Part 5 of the Performance-Aware Programming series. Please see the Table of Contents to quickly navigate through the rest of the course as it is updated, and the code repository for downloadable code listings.

Read more

The questions addressed in this video are:

• [01:02] “Could you suggest a resource to learn how to work with large datasets that don’t fit into VRAM, or even regular RAM?”

• [07:24] “I do not know how this compares to other approaches, as I’ve not really tried to optimize my profilers much as it’s not really my main area of work, but your profiling explanation made me think of a lock-debug profiler I built. It sounds similar to me at least. Do you agree?

  In my solution, each worker thread owns its profiling buffers completely. I keep a thread-local pointer to a block-based buffer that grows in chunks when needed, and all writes are done by the thread with no locks. The only shared step is that when a thread creates its first block, it registers it once by pushing it into a global linked list so the UI can later iterate all threads. To avoid stalling the writers, I double-buffered it. Each thread allocates two buffers and writes into one of them.”

• [09:46] “About memory usage and the program stack, I believed that this stack could more easily fit in the L1 cache. Isn’t there a higher risk of cache misses at that level if the data to use has been allocated elsewhere? Would it even be noticeable in terms of performance?”

• [14:32] “What’s the best way to multithread a software rasterizer? In my experience, scan line interlacing per triangle was horrific.”

• [18:18] “Do you know why in the file processing test on the same machine on linux mmaping the file could be outperforming all other methods on medium and large mapped chunk sizes, and on windows the same test was worse than everyone else?”

• [21:18] “Regarding callbacks, I think I understood the argument about moving things to a queue instead. However, I also can see the benefit of a callback because that’s synchronous. What if the file IO read some data and stored in a buffer, and the callback now is supposed to handle that read data. If instead you move things to a queue, you’d have to keep allocating new buffers to keep working on those asynchronous reads, because you don’t know when the client code will handle those reads. If instead you can very quickly handle this buffer of read data synchronously in the callback, the file IO can reuse the same buffer to store the next chunk.”

• [25:45] “Can you steelman cases for which an arena/bump allocator (are these the same thing?) is not the preferred way to allocate memory (I imagine it is when lifetimes are not apriori known, but perhaps I am missing more subtlety)? In such cases, what is your preferred method of allocation? Are you forced to go back to new/delete?”

• [33:01] “Re: Dead Code Elimination Prevention Macros, I’ve got identical results with `asm volatile (”“ : “+v”(Value));` which tells the compiler that `Value` is both input and output, forcing it to initialize it, as well as preventing from assuming a specific value. It would generate `vpxor` for `0.0` and `vmovaps` for `0.5`. The advantage here is that it’s quite generic and doen’t depend on operand size/type.Could you please elaborate on your choice of explicitly using instructions?”

• [35:14] “Concerning callbacks, is there a benefit to use callbacks for print-outs/messaging? We have a part of the program that do some calculations which can take time, and it uses callbacks to notify the user about the progress, and waiting to the end of all the computations is not good enough. We also use a callback for a type of mesh calculation that depends on things that this other part of the program shouldn’t know about. (These things were not my decision, but I guess making the module as isolated as possible makes it easier to use it as a module in another program)”

• [37:42] “How do you determine if a solution to a problem is more complex than it needs to be? And for inherently complex and interconnected problems, how do you determine if it needs to be subdivided or not? Is there a general approach for working on a complex system with lots of moving parts? (other than me complaining about it :) )”

Read more

This is the eighth video in Part 5 of the Performance-Aware Programming series. Please see the Table of Contents to quickly navigate through the rest of the course as it is updated, and the code repository for downloadable code listings.

Read more

The questions addressed in this video are:

• [00:04] “What are your top 5 recommendations for must have projects or applications to build at least once in your life to make you a better programmer ?”

• [14:32] “My question is regarding your previous comment on stack below, where you mentioned that we don't want to do a lot with the stack for performance oriented code. Before this course, I had impression that I should prefer stack allocation to heap allocation (maybe just C++) because I read/watched somewhere saying that 1) stack allocation/dealloc is faster (just move the stack pointer) than heap allocation 2) stack object has more obvious lifetime, 3) stack has no fragmentation concern which improves locality 4) you call on stack object directly while heap object involves indirection.

  I wonder whether my understanding above is correct or not. If not, does that mean I can prefer using heap over stack allocation most of the time? Could you please further comment? Thanks!”

• [32:48] “One idea is to avoid dynamic allocation in the critical path by pushing each measurement into a dedicated logging/profiler thread through a channel. The worker thread would record measurements while the computation threads only perform the minimal ‘send’ operation. But atomic operations, queues, and cross-thread communication can also add overhead, but also distort the original program execution … I’d like advice on how to judge whether this dedicated-thread idea is sound, and in general how to think about designing a low-overhead profiler.”

• [44:20] “What is your opinion on using callbacks for say signal handling? They often seem necessary but I may be overusing them, which leads me to believe my overall architecture could be flawed. But generally speaking, what’s your view on callbacks? Love them or hate them? Do you try to avoid them?”

Read more

This is the seventh video in Part 5 of the Performance-Aware Programming series. Please see the Table of Contents to quickly navigate through the rest of the course as it is updated, and the code repository for downloadable code listings.

Read more

The questions addressed in this video are:

• [00:05] “Question regarding the ‘fat struct’ approach: do you ever find yourself thinking about excess memory consumption caused by some entities having unused fields?”

• [04:29] “About fat structs, you said we would initialize the struct to be able to be A or B depending on the situation, if you imagine that fat struct to be of type Result with cases Success or Error, how would you be able to initialize the Error in the case you have a Success as there was no Error, and vice-versa. This type for example exists in F# and it makes sense to initialize Success only or Error only.”

• [07:40] “Do you have any suggestions for an off the shelf tool to measure bandwidth and flops for a machine your run it on?”

• [13:10] “I sense a recurring theme of reliability and predictability – preferring simple control flow to early returns, preferring simple compilers to strict aliasing, preferring large blocks of memory to pointer festivals, etc. You’ve also spoken about improving _robustness_ by preferring zero/dummy values that just flow through code to null pointers, and preferring handles to pointers (from the iCloud example we just saw).

  What do you mean with “robustness”, and what other techniques can I use to make my code more robust in this way?”

• [18:00] “Do you have any thoughts on Apple’s approach to SIMD?”

• [20:10] “Hi Casey, on the recent podcast with Marco you said that if you could choose a single piece of software to be magically redesigned it would definitely be the browser because the software platform it defines is bad. Could you please elaborate on that? What are the main problems with today’s browsers from your perspective?”

• [22:30] “I’m a bit confused when analyzing the bandwidth I get when reading directly from the volume (using ReadFile with the path ‘\.\C:’ with an offset, for example). As far as I know, that’s the correct way to read system files like the $MFT (which I can currently read properly, by the way).

  When reading 20 GB of contiguous data from the beginning of the volume, I get 2.6 GB/s, and it doesn’t matter whether I use the FILE_FLAG_NO_BUFFERING flag or not—the result is the same. I’d expect something closer to a non-cached read (4.9 GB/s), but I’m getting the same throughput as a cold cached read. I’m not sure where this penalty comes from (assuming the read isn’t triggering extra cache operations since it’s non-buffered).

  Any idea what might be going on here? Do you think these read bandwidths make sense?”

• [25:46] “Will we be able to reuse the coefficients we’re currently using for f64 sine for f32 sine or will we need new ones?”

• [28:24] “I have a question about PC hardware components. I’m not clear on things like the motherboard and chipset. Do they play any role in performance? They vary a lot in price even with similar features, so I assume some aspects must affect overall system performance. Could you give a brief overview of these system parts, if possible?

  Or, to rephrase my question: When you’re building a PC, what do you specifically look at besides just the CPU, RAM, disk and GPU in terms of performance? How do you decide what’s suitable for your specific builds?”

• [36:09] “In my code, I ended up having a single loop over the input that directly produced the haversine sum, rather than splitting parsing and math into two loops. But that means if I want to time parsing vs math, I have to put blocks into the loop, which (seemingly) inevitably introduces a lot of overhead.

  Is there a good way to handle this? The best way I can think of is to instead temporarily comment out parts and just time the rest, though while that’s easy to do with the math part, it seems harder to do for the parsing part, since you still have to somehow produce dummy data for the math while making sure this doesn’t lead to any compiler optimizations you wouldn’t otherwise get.”

• [39:28] “Sorry for repeating the question from the last Q&A, but here is: I have just gotten to it, did it and looked up your solution in QA47 to cross reference. There is one thing that we got differently, and I don’t quite understand your reasoning about it:

  You said that shl rbx, 0 should be recognized by the frontend as a nop and not do anything with flags, but would produce rbx.

  1) If the frontend sees it as a nop, why would it RAW the value of rbx, and not just be a pure nop?

  2) I actually thought that it would not be recognized as a nop (I didn’t find anything about this kind of optimization, i presumed it would be somewhere near zero idiom stuff in the manual), and then it seems like shl will have not only a RAW on rbx, but also on all the flags, as it has be ready that the ALU will say that the shift was 0 and the previous value of flags should be preserved (i. e. RAW)

  So the question is, why is it that rbx is a RAW and flags are skipped, and do you know if there is any place in the docs where such a frontend optimization might be mentioned?”

Read more

This is the sixth video in Part 5 of the Performance-Aware Programming series. Please see the Table of Contents to quickly navigate through the rest of the course as it is updated, and the code repository for downloadable code listings.

Read more

The questions addressed in this video are:

• [00:00:09] “How far can we take SIMD use? What if we put a way bigger onus on eliminating branching and developing parallellizable code? The language K, the simdjson library, the Co-dfns compiler, and the new Box2D engine all blast competitors out of the water by thinking of data parallelism. SIMD use is arduous now - should we start designing our languages and APIs around it? What’s the path to broader cross pollination as an industry? It really seems an untapped potential, right?”

• [00:04:01] “Could you give examples of the kind of substrate work you’re hoping more people take seriously? Are you referring to teams like the WSL or Visual Studio performance teams, or something even deeper (or higher-level)? You’ve cited companies rewriting websites for performance and Microsoft’s console output claims - are these the kinds of things you mean? And if you mean something different, how do you think someone can get involved in that kind of work? I’m trying to understand the bigger vision. Is it that everything on the internet runs as fast as the McMaster-Carr Supply Store website?”

• [00:07:45] “When you critique tools like Git, are you expressing a personal frustration, or do you think there’s an objectively better way these tools could work? It’s hard for me to imagine a world where I don’t need to memorize Git or AWS minutiae - are there products today that represent what you think “just working” should look like?”

• [00:13:00] “What do you think about coroutines?”

• [00:15:50] “Will a VOD of the ‘Research Overview for “The Big OOPs”’ livestream be made available later?”

• [00:16:10] “Do you find that this performance aware development you are talking about also improves the quality of the software in general? Is there any connection? Further, once you decide that something needs to have its own test or tests, what techniques(as in test code organization, handling of test data etc.) and tools you find useful in creating those tests?”

• [00:23:18] “I’m currently onboarding at my first ever job. It’s an giant legacy PHP codebase which primarily uses OOP. I don’t think OOP is great. But if you were forced to use classes for everything, what would be the best way to do it?”

• [00:26:40] “It’s interesting how I get much better performance on Zen3 on Linux, than reference haversine even for the replacement case. ”

• [00:30:47] “I’ve been watching nearly all the BSC talks, and they’ve made me realize I might have a wrong understanding of what a type is. What would be the best definition?”

• [00:35:19] “It seems like there has been a push in recent years towards languages with stronger type systems and static analysis (such as Rust’s borrow checker). Do you think that this trend meaningfully improves software quality, and if so what static analysis tools (both existing and hypothetical) do you think would be the most beneficial for a performance-minded programmer?”

• [00:40:35] “I watched a YouTube video about the Montana mini-computer, and I understood how the concept of a function is implemented at the assembly level. I was wondering: what is a virtual function as defined in a high-level language, and how does that translate to a CPU? Along the same lines, I didn’t fully understand the concept of volatile. It seems to be related to the stack—could you explain how a volatile variable is represented at the assembly/CPU level?”

• [01:04:10] “With Intel AMX becoming more mature or widely-known about, do you think it is or will be possible to start doing things typically done on GPUs (texturing, filtering, convolving, etc) on CPUs in the future? Assuming it will be, do you think GPU vendors will finally start opening up a lá the 30 million line problem in a fight to remain competitive with CPU vendors?”

• [01:13:00] “Unlike games, a lot of the value in the Apple world comes from tight integration with all of their ecosystem and design language (ex: iphone widgets, watch now-playing view, siri searchability, sharing to other apps or airdrop, accessibility integration, etc...) But of course Apple is has a lot of OOP style and “declarative frameworks,” which means that I wouldn’t have control over the code that actually runs these features.

  How can I still use a handmade philosophy of writing my own simpler more focused code rather than depending on lots of slow and volatile libraries?”

• [01:15:36] “do you have any advice for gracefully avoiding or recovering when Apple helpfully deletes your stuff in the background (kills your process when you switch apps, makes you redownload files from icloud)”

• [01:19:21] “In your talk at the Better Software Conference you mention the “fat struct” as a good default option for programming in a systems level language. I think I know the gist of what you mean by this, but I am curious if you have a slightly more formal definition for the term and a general explanation for why it’s a good default approach.”

• [01:27:32] “Most of the course so far talk about programs and assembly for actual chips. How do the aspects of concern under performance aware programming change, if at all, if the target is WASM?”

• [01:30:13] “I perfectly understand why writing to al in a loop is slower than writing to rax, but I get different results for al, ax, eax and rax. The loop writing to al and ax goes at nearly 1/4 of the loop writing to rax, but the eax loop goes at 1/2 that speed. Shouldn’t they run the same since writing to eax does not preserve the upper bits?”

• [01:31:01] “If there’s really no ‘rax’ in a cpu at any given point of time, why (how?) debuggers show a single value what you stop them, or linux will only write a single value in the core dump file? Shouldn’t there be some kind of a tree? I just don’t know if I can’t trust this information for debugging... What if it shows a register value from one branch, but the bug was caused by the value from another, won’t I be misled?”

• [01:40:21] “I was trying to reproduce your results from the RAT and register file lecture. I am running them on Alder Lake chip (i7-12700H). My results were quite the opposite to yours the add only loop was either having similar performance or run much faster compared to mov and add one. I found your article that hints that Alder Lake is able to decouple those chained adds.”

Read more

This is the fifth video in Part 5 of the Performance-Aware Programming series. Please see the Table of Contents to quickly navigate through the rest of the course as it is updated.

Read more

The questions addressed in this video are:

• [00:03] “Hi, I've seen you talk before about the state of gpu apis, and I am aware that you were talking about SoC solution to this and many other problems with current computers. My question is this, how would you design a gpu api, if you had a magic stick that you could shake to make it appear and be common on all computers today? If I understand your position, you'd have the programmer talk directly(or at least through a paper thin layer) to the gpu, but how would that work in practice? If that's not too much to ask, I'd love seeing a short snippet of pseudocode of how it would work on the consumer side, thanks!”

• [04:10] “When dealing with async io, do you think an await style interface like in javascript/go is a good model, or do you think some simple state machine or any other method is better? I find that writing state machines for this kind of stuff gets overly explicit in a lot of cases. Thanks!”

• [07:52] “Given the course, I was trying to apply some techniques to my own toy problem that given a list of words and an NxN grid tries to generate a wordpuzzle where each word is added horizontally, vertically or diagonally.

  I tried to reduce pessimization as much as possible. The program performs a recursive search where each word added successfully to the grid increases the depth by one. Memory is pushed and popped with a single memory arena of max 512KiB. The hot code is the check that given a word and the current board, to see if the word will fit.

  And I am having a hard time vectorizing this loop such that it's actually more performant. The single byte checks seem to outperform vectorization by a factor 10 as the data is too sparse? I also could not find an AVX "scatter" function that does the opposite of a movemask_epi8. Was wondering if you have any thoughts on how one would optimize this further.”

• [13:03] “I was making a homework about asm volatile, and I noticed that all fma instructions were using memory operands and now I'm wondering why there are no fma instructions with immediate operands? Surely it should be beneficial to bake values in some cases, right? Or maybe there are never immediate operands for floating point instructions.”

• [17:54] “Will estimating the cost of more ‘branchy’ workloads like lexing or e.g. json parsing be covered?

  For context I am trying to apply what I've learned from this course to optimize a lexer for a programming language and have gotten from ~0.8 GB/s to ~1.3 GB/s lexing the linux source code. However, it seems impossible for me to get it to run any faster, eventhough 1.3 GB/s is nowhere near memory bandwidth and most of the work is just deciding what kind of token to spit out and how much to advance. It feels to me like ~2GB/s or so could be the limit to how fast you could lex one token at a time, and going above that would require producing more than one token like I believe simdjson does. However, I have no clue if this is remotely correct, since my intuition about the cost of branchy code is provably very bad.”

• [20:12] “Do you have any good resources that outline how to improve a user's experience. The course emphasise performance but I'm curious what other things would you consider to improve a user's experience.”

Read more

This is the fourth video in Part 5 of the Performance-Aware Programming series. Please see the Table of Contents to quickly navigate through the rest of the course as it is updated. The listing referenced in the video (listing 196) is available on the github.

Read more

The questions addressed in this video are:

• [00:02] “This is more of a learning personal project and I'm looking for something I can do that has a little bigger impact. Do you have some other tool/program/improvement you'd like to see but don't have the bandwidth to do yourself that you can share or just an avenue you'd like to see explored more that I could look into to produce something of value to other fellow programmers?”

• [02:57] “Hey Casey, I posted a question about non-text programming languages and fear I was a little late on the Q&A cycle, Substack didn't send me an email for it :( , for the question to make it into this Q&A's rotation. I am still very curious what your thoughts about a non-text language are, or if you have talked about them before and have a link to those articles/videos!”

• [03:51] “I wanted to know if you have already covered ‘False Sharing’ from a cache point of view and its impact on performance. If not, do you have any plans for it? Do you plan to also cover effects of NUMA on performance?”

• [08:05] “I just rewatched your video on the GJK algorithm from 2006, and when you talk about the triangle case you say that, although there are six ifs and elses written down, you only ever execute three of them at most, so it's at most three tests and jumps. Since the logic only relies on dot products and cross products, I would assume that the multiplies and the adds would have a lesser impact on the performance compared to the penalty of mispredicting multiple branches so close to each other.

  Am I getting something wrong?”

• [11:50] “According to you, what would be the impact of AI on performance aware programming? I mean, is it likely to become more common that organizations / individuals offload this task to sophisticated AI coding agents? I know its a very vague query with possibly no clear answers but just wanted to know what this community thinks about it.”

• [17:34] “Hi Casey, I was working on a simple CSV reader to analyze some logs but I have many huge files and it takes a while. I wanted to try out Intel V Tune and it says that the code is mostly front-end bound and has 100% DSB misses. I was wondering what are those? Are they a problem? How can we solve them?”

• [23:10] “Why is memory management so obscure, and why are people inventing so many languages to ( ostensibly ) fix that, while criticizing C for being unsafe? Why are use-after-free / double-free and so on, issues mentioned when talking about C being unsafe? In other words, why would I need to free memory, when I know exactly how much memory a program will ever need and I can keep on reusing it ( you can't know at a given time how much memory you need, but there is a limit, since you can't allocate infinite memory ).

  In real life, when we discover new concepts ( like in physics ) we don't invent a new language to explains those concepts in., instead, we improve our current language / language practices.
  No worries if you can't include everything from the msg in the Q&A.”

• [29:33] “Hi Casey, before taking the course I would consider using SIMD instead of scalar operations and utilizing memory caches properly to be optimizations. But it seems like you prefer to think about them as a performance aware programming. Then I wonder what would be some concrete examples of optimizations? I just can't imagine something beyond that to get even more performance.”

• [32:59] “Hello, does anybody have any idea why could there be a huge difference between reading a file to mmap'ed memory and malloc'ed memory on Linux, AMD Zen3 chip?”

Read more

This is the fifteenth video in Part 4 of the Performance-Aware Programming series. Please see the Table of Contents to quickly navigate through the rest of the course as it is updated. The listings referenced in the video (listing 194 and 195) are available on the github.

Read more

The questions addressed in this video are:

• [00:02] “Not the original author but I have a follow up question on the BTS/bits thread. On x86, writes are atomic only with byte granularity if I'm not mistaken, so in order for BTS to work properly and have multiple threads toggling bits _on the same byte_, there needs to be some sort of atomicity baked in for the RMW series of operations. Seeing the hand rolled series of operations doesn't have any lock/atomic op in there, wouldn't it have a bug when multiple threads try to toggle bits on the exact same byte?”

• [08:00] “I have only dabbled in low-level OS and/or bare-metal hypervisor code, but when I did, I was surprised to find how much setup was required to get multiple cores working in x86. Surprisingly little was set up out of the box - it seemed that the cores all started off running the same code and accessing the same data, and I assume that each core's registers started off in the same state (not sure about the details). So, what are the main considerations to get from that initial blank state (post BIOS/entering 64-bit mode) to where your OS/hypervisor can have multiple cores running separate threads of code (and maybe scheduling them)?

  A related question, at the level of OS applications: You mentioned once that you prefer to roll your own multithreading code instead of relying on libraries. Having only ever used threading libraries in various languages, it's not clear where to even start, or how my hand-rolled threading code would even benefit me over generic threading libraries. What are the main considerations for rolling my own threading code instead of using something like pthreads? Or am I confusing threading libraries with OS-level threading APIs?”

• [18:12] “As a developer with 10 years of experience in Java, I’m considering a career move—not due to layoffs, but in search of a more fulfilling role. (I am also considering other languages not just Java)

  I know companies like RAD Game Tools are rare (if not unique), so I’m curious:

  Where would you recommend looking for opportunities that prioritize meaningful work?”

• [25:26] “I imagine that you don't write your own math implementations for all your projects, since you emphasized that we do this for educational purposes. What do you actually use?”

• [29:36] “I recently had an assignment for a 2D graphics course at my university, where i had to implement a edge detection filter in python and c. The goal of the assignment was to compare the performance and later implement a Cython version of the function. I implemented a AVX version and the professor was super stoked. He told me that he used to fiddle with MMX but that it was always a pain to have it compile and that it never ran on other machines because of incompatibility. Was it really that bad back in the day? I feel like even AVX2 now has pretty wide spread support on most computers today, and getting something to compile is just one compiler flag. How do you deal with different architectures / different feature sets to make it more convenient to program? Do you have a wrapper? If so, would you mind sharing it?”

• [34:57] “I finally put together a minimal reproducer for the bts question.”

• [35:25] “What is the performance difference between the original haversine versus the new haversine with your own sin/cos/etc functions? I feel like that was missing from the last video”

• [36:00] “If you could design your own computer science undergrad program, what sorts of classes would you include and focus on and what would you change from the way current universities do things?”

Read more

This is the fourteenth video in Part 4 of the Performance-Aware Programming series. Please see the Table of Contents to quickly navigate through the rest of the course as it is updated. The listings referenced in the video (listing 192 and 193) are available on the github.

Read more

This is the thirteenth video in Part 4 of the Performance-Aware Programming series. Please see the Table of Contents to quickly navigate through the rest of the course as it is updated. The listings referenced in the video (listing 190 and 191) are available on the github.

Read more