Rendered at 19:46:53 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
Dwedit 5 hours ago [-]
C++ ABIs are why Win32 settled on things like COM to do cross-module passing of objects. Because nobody could agree on a standard, the COM standard enforced a very specific calling convention, and a very specific vtable layout for COM objects.
pjmlp 5 hours ago [-]
Not only, there is also XPC and IO/DriverKit on Apple, Android IPC, D-BUS, FIDL on Fucshia,....
Also C ABI also does not exist, people keep mistaking the ABI of their favourite C compiler with the OS ABI, which only overlap if the OS was written in C to start with.
For example on mainframes and micros, naturally not written in C, it is either a bytecode based ABI like TIMI on IBM i, or language environments like on z/OS, ClearPath MCP, OS 2200 and so forth.
And as a reminder, from a famous WG14 and WG21 member, and former Rust contributor,
And Delphi came along and used the same layout, so an interfaced Delphi object has beautiful COM interop.
pjmlp 2 hours ago [-]
Which is also used by C++ Builder.
In a tragic way, given the prevalence of COM in Windows, especially since Vista, someone at Microsoft should offer a few Delphi and C++ Builder licenses to the teams responsible for doing COM tooling.
Because MFC/OLE, ATL, WTL, WRL, C++/WinRT all have their sharp edges and could be so much better, if someone actually cared about productivity and framework ergonomics.
WalterBright 3 hours ago [-]
> and a very specific vtable layout for COM objects
Fun fact: Microsoft developed COM based on how Zortech C++ virtual functions worked. (It predated Microsoft C++ by years.)
throwlifeaway 2 hours ago [-]
You mean... using a vtable? Are you claiming to have invented the concept of vtables?
quotemstr 4 hours ago [-]
And COM enforces lifecycle rules, transparent remoting, uniform activation (constructors basically), security primitives, marshalling, transparent async calls (yes, you can call any method asynchronously), and a ton of other things. It's actually very good and it's a shame most don't know about it and most of the rest sneer at it.
dblohm7 4 hours ago [-]
I agree!
(Though you cannot call just any method asynchronously; the interface's IDL needs specific annotations and the underlying object needs to implement ICallFactory.)
pjmlp 2 hours ago [-]
Which shows how many devs never learnt about Windows development past Windows XP, given that COM is nowadays the main API delivery mechanism, as they redid Longhorn ideas in C++.
rramadass 4 hours ago [-]
Well said.
COM was one of the best realizations of "Component-based Software Engineering" and Brad Cox's "Software-IC" model. It was a binary standard and so components could be written in any language and yet be assured of perfect interoperability provided you followed all the rules/conventions. There was a bunch of boiler-plate for the framework itself but once you understood it, everything was smooth sailing in your language of choice.
One of the things i always advise people is not to focus only on the current way of doing things but to study older well-known libraries/frameworks/architectures/kernels/etc. to really understand "Software Engineering" from many perspectives. That is where insight comes from and real understanding happens.
For people interested in understanding COM, see the classic Essential COM by Don Box.
2 hours ago [-]
pjmlp 2 hours ago [-]
COM is heavily used in Windows today, no need for past tense.
rramadass 2 hours ago [-]
Not directly by app developers. Everybody uses only WinRT/.NET based managed languages/runtimes/libraries/frameworks etc.
pjmlp 2 hours ago [-]
Who uses WinRT besides Microsoft employees, or APIs that are only available in WinRT like MIDI 2.0?
Regardless, WinRT is COM.
It only adds IInspectable as additional interface alongside IUnknown, and .NET metadata files instead of type libraries.
rramadass 1 hours ago [-]
I said "WinRT/.NET based managed languages/runtimes/libraries/frameworks" implying that it is the higher-level constructs that are used by app developers.
If you were to interview any "standard" windows programmer today i can almost guarantee they know nothing about COM even though WinRT itself is an evolution of COM and .NET also provides COM interop. Only the older senior programmers who have used it directly know of it.
pjmlp 1 hours ago [-]
Ignoring C++ devs on purpose?
Also no one uses WinUI, if that is what you are implying with WinRT, only Microsoft employees on Windows team forced to deal with that clusterfuck framework after Project Reunion pivoted into WinAppSDK.
What do you think Qt, VCL, FireMonkey, ImGui, wxWindgets, JUCE make use of?
mohamedkoubaa 5 hours ago [-]
Given OOP, the only ABI for an object is a vtable and that is left unspecified (for good reason) by the C++ standard.
COM said, "objects are cool and modules are cool so we need an ABI for objects" and specified a vtable. It isn't the only programming model which continued from the same observation: Python did too with its absolutely abysmal leaky PyObject ABI
pjmlp 4 hours ago [-]
SOM predates COM, and is even cooler, as it supported class inheritance, metaclasses and Smalltalk as target language.
On OS/2, Smalltalk enjoyed a role similar to what would be .NET and VB on Windows.
Sure you can use delegation with aggregation, and type libraries (nowadays .NET metadata), but still isn't as ergonomic.
rramadass 4 hours ago [-]
COM designers were inspired by C++ but their goals were different. They were designing a language-neutral binary standard (aka ABI) and hence took C++'s approach and tweaked it for their specification.
I understand that Move Semantics led to better implementations and that this is why things changed. I know that in this case, problems arose when transitioning from C++98 to what's commonly called Modern C++ (C++11). However, I'm not very familiar with what the specific issues were with COW. If you happen to know of any documents that describe this problem in more detail, I'd appreciate it if you could let me know.
I'm sorry for always asking questions. I've looked up a few documents, but since you clearly know much more about this than I do, I'm asking to study it myself as well. If it's too much trouble, feel free not to reply.
aw1621107 4 hours ago [-]
The changes to std::string were made because "the [pre-C++11] definition of basic_string allows only very limited concurrent access to strings. Such limited concurrency will inhibit performance in multi-threaded applications." The proposed fix was to make "all iterator and element access operations safely concurrently executable." [0]. This involved limiting the circumstances in which iterators could be invalidated, which among other things meant that non-const operator[] could no longer invalidate iterators, which meant copy-on-write was no longer a viable implementation [1].
Finally, C++11 tightened the specification for std::string viz. contiguous memory, some methods must have O(1) access time, when references/iterators/pointers can/cannot be invalidated etc. which doomed COW.
jdw64 2 hours ago [-]
I'm sorry for always bothering you, senior. I've learned quite a lot from the books you recommended. I've especially been studying low-level topics lately, and the books you suggested have been very helpful. Have a great day.
jeffbee 4 hours ago [-]
One nice thing to remember is for projects you control, you can dictate the ABI. There isn't a law enforcement bureau that will arrest you for passing parameters in whatever register suits you. You just have to imagine life differently than they did in the 70s when they dreamed up dynamic libraries for reasons that are largely no longer valid.
rramadass 3 hours ago [-]
This technique is often used for security-sensitive projects involving binary protection, obfuscation etc.
PS: James Coplien in his excellent Advanced C++ Programming Styles and Idioms shows many techniques one of which is actually replacing vtable entries at runtime to mimic features from more dynamical runtime languages (eg. Smalltalk).
jdw64 5 hours ago [-]
Does the C++ standard guarantee binary ABI? Isn't that something that compiler and library contributors handle separately? So I think the arguments you see online are more accurate—vendors are the ones maintaining it.
In other words, the ABI we rely on today isn't really part of the C+ standard—it's more like the Itnaium C++ ABI or the MSVC C++ ABI.
In the end, I think the ABI stays stable because of community conventions established by compiler vendors.
jcranmer 5 hours ago [-]
No, ABI is not part of the C++ standard, although there are parts of C++ that do go "this is an ABI thing" (e.g., [[no_unique_address]]).
ABI is more of OS-level thing. Most systems these days follow the SysV ABI, which is largely defined by the hardware manufacturers via the processor-specific supplements (the x86-64 one is here: https://gitlab.com/x86-psABIs/x86-64-ABI). These largely delegate C++-specific conventions to the Itanium C++ ABI (which they likely directly reference), although the ARM ecosystem uses a somewhat different layout for the exception handling tables. Microsoft uses a different ABI for both the underlying C ABI and for the C++ compatibility layer built on top of the C ABI.
One of the issues that crops up is that vendors end up needing to add extensions to the ABI for various reasons, and these extensions tend to end up being incompatible, since they're added before they've had a time to be standardized. 16-bit floats is a particular historical bugbear, as is the C23 _BitInt stuff.
jdw64 5 hours ago [-]
Thank you for the kind comment. I'll make a note of it and look into it.
pjmlp 5 hours ago [-]
No, neither does the C standard.
With exception of Swift, D, and bytecode based languages, the ABI is left to the vendors.
For those that think ISO/IEC 9899:2024 PDF has anything related to ABI, the actual ABI used by C compilers, is the OS ABI, if the OS was written in C to start with, and naturally this overlaps quite nicely with UNIX like OSes, and Windows.
It isn't like that on other platforms that decided to either use other systems languages, or expose their OS APIs in a different way, e.g. mainframes, micros, Android, ChromeOS, WebOS,...
Someone 2 hours ago [-]
> With exception of Swift, D, and bytecode based languages, the ABI is left to the vendors.
I don’t see exceptions there. Apple defines the ABI of Apple’s Swift’s implementation, Walter Bright (or his team) defines that of D, Python defines its ABI, etc.
If you were to write a Swift/D/etc compiler, you’re free to define your own ABI. Disadvantage is that you would give up linking with code compiled by the other compiler (workarounds such as pragmas, C++’s extern "C", are possible)
pjmlp 2 hours ago [-]
You're missing who defines the language standard.
You're free to do whatever you feel like on your implementation, including not being compliant with the official language standard.
mohamedkoubaa 5 hours ago [-]
Isn't this one reason why C is the de facto FFI? OSes happen to use it in their OS ABI and so everything is forced to speak it
bluGill 4 hours ago [-]
One reason. The other is C is easy. Any language can implement it and have confidence it can work. (there are other options that are just as easy, but C won). C++ is much harder because of things like function overloading and exceptions - none of it is difficult, but there is a lot more to do and thus more opportunities for bugs.
uecker 8 minutes ago [-]
Easy and still expressive, so you can map many things to C. And while the language standard does not define an ABI it takes a lot of care not to require any ABI break.
mohamedkoubaa 3 hours ago [-]
I would go far as to say that modern C++ is an Eldritch horror of a programming experience
pjmlp 3 hours ago [-]
Unfortunately no one is going to rewrite LLVM, GCC, CUDA, DirectX, Metal, SYSCL, V8,... into something else.
Same applies to C and UNIX/POSIX, OpenGL/Vulkan/OpenCL/NVN/LibGNM(X),...
Thus analysers and language subsetting tools, enforcing language style guides and safer coding practices.
rramadass 2 hours ago [-]
Oh come on; that is BS and you know it.
It is complex/complicated/baroque but for the power, flexibility, real-world-usage that it provides it is worth the effort. The mistake that people make is to try and learn it all at the same time which overwhelms them.
Use a good book like Discovering Modern C++: An Intensive Course for Scientists, Engineers, and Programmers by Peter Gottschling and you should have no problem at all.
mohamedkoubaa 60 minutes ago [-]
For context: I have over a decade of professional experience with c++ in large scale software projects and it is by far the language over which I have the most mastery.
pjmlp 4 hours ago [-]
OS written the C, which is what many overlook.
If you are on e.g. z/OS you would be using ILE, Integrated Language Environment, on ChromeOS JS/WASM, on Android either DEX or JNI,...
andyg_blog 5 hours ago [-]
It doesn't, and FWIW the article says that. A meta note: ABI stability is a holy war inside the C++ community. Those against it argue that it's holding back real language evolution. Herb Sutter is even working on his own C++ offshoot that shows what could be.
Mond_ 5 hours ago [-]
Afaik Herb Sutter largely stopped working on Cppfront a few months ago, and was never quite supposed to be a real thing, just a personal playground (or so Sutter claims at this point, anyway).
Afaik Carbon is at this point the only attempt at a successor language that's still going?
pjmlp 4 hours ago [-]
Probably, but note that Google says they are they main customer anyway, and it is an experiment above anything else.
Anyone that can reach out to Rust, Go, Java, C#, whatever, should do that preferably.
There are also some efforts to tame existing C++ with profiles, and replacing UB with erroneous behaviour, and that's about it.
bluGill 4 hours ago [-]
The C++ standard doesn't say anything about binary ABI (as you say generally either Itanium or MSVC, though there are other options they are rare). However the people who write that standard are very sensitive to the vendors and users of C++ who depend on a stable binary ABI. Thus they take extreme care to ensure that no change to the standard breaks binary ABI.
The C++ standard did force gcc to break the ABI of std::string (copy on write was banned - for good reason). They then watched the gcc community work through 10 years of pain to make the transition. They are also well aware that python 3 broke compatibility with Python 2 - and again it resulted in 10 years of pain for the python community to deal with that. With this history there are a lot of experts strongly against any breaking change. It might happen anyway, but only with strong justification and likely an attempt at a migration of some form (what? there are a lot of examples of migration plans that don't work that they are aware of)
Again, it is not because of convention, it is because of painful experience from those who break it.
MFHava 3 hours ago [-]
> They then watched the gcc community work through 10 years of pain to make the transition.
During the same time the MSVC compiler broke ABI multiple times and the world didn't end.
The fact that breaking ABI was such a shitshow for libstc++ is more related to the way C++/ABI/SOs is/are handled in Linux...
plorkyeran 3 hours ago [-]
In the time since that VC++ has switched to a frozen ABI model that’s much closer to what Linux distros have. Each version of VC++ being ABI-incompatible and requiring each application to be built with a single version and ship its c++ runtime had some significant pros and cons.
Microsoft never said it outright, but I’ve always assumed the major con that made them switch was that people didn’t buy new Visual Studio versions because upgrading VS required also upgrading all of your binary blob closed-source dependencies.
mohamedkoubaa 5 hours ago [-]
C++ famously has no ABI, it is the implementations of C++ that have ABIs. The reason the situation is so bad is that it's considered explicitly out of scope of the language (it targets an abstract machine).
leni536 5 hours ago [-]
Neither of the ISO C and ISO C++ standards define an ABI.
mohamedkoubaa 4 hours ago [-]
Right, C doesn't specify an ABI either but (many) OSes do and they provide their ABIs in C so C became the defacto ABI of a given platform.
leni536 4 hours ago [-]
Yes and no. The Linux kernel barely interacts with the C calling convention ABI (vDSO), and you can interact with the OS without going through a C library. I guess the layout of some structs are exposed through some binary protocols, but most of them are not C function calls.
Also C ABI also does not exist, people keep mistaking the ABI of their favourite C compiler with the OS ABI, which only overlap if the OS was written in C to start with.
For example on mainframes and micros, naturally not written in C, it is either a bytecode based ABI like TIMI on IBM i, or language environments like on z/OS, ClearPath MCP, OS 2200 and so forth.
And as a reminder, from a famous WG14 and WG21 member, and former Rust contributor,
"To Save C, We Must Save ABI"
https://thephd.dev/to-save-c-we-must-save-abi-fixing-c-funct...
In a tragic way, given the prevalence of COM in Windows, especially since Vista, someone at Microsoft should offer a few Delphi and C++ Builder licenses to the teams responsible for doing COM tooling.
Because MFC/OLE, ATL, WTL, WRL, C++/WinRT all have their sharp edges and could be so much better, if someone actually cared about productivity and framework ergonomics.
Fun fact: Microsoft developed COM based on how Zortech C++ virtual functions worked. (It predated Microsoft C++ by years.)
(Though you cannot call just any method asynchronously; the interface's IDL needs specific annotations and the underlying object needs to implement ICallFactory.)
COM was one of the best realizations of "Component-based Software Engineering" and Brad Cox's "Software-IC" model. It was a binary standard and so components could be written in any language and yet be assured of perfect interoperability provided you followed all the rules/conventions. There was a bunch of boiler-plate for the framework itself but once you understood it, everything was smooth sailing in your language of choice.
One of the things i always advise people is not to focus only on the current way of doing things but to study older well-known libraries/frameworks/architectures/kernels/etc. to really understand "Software Engineering" from many perspectives. That is where insight comes from and real understanding happens.
For people interested in understanding COM, see the classic Essential COM by Don Box.
Regardless, WinRT is COM.
It only adds IInspectable as additional interface alongside IUnknown, and .NET metadata files instead of type libraries.
If you were to interview any "standard" windows programmer today i can almost guarantee they know nothing about COM even though WinRT itself is an evolution of COM and .NET also provides COM interop. Only the older senior programmers who have used it directly know of it.
Also no one uses WinUI, if that is what you are implying with WinRT, only Microsoft employees on Windows team forced to deal with that clusterfuck framework after Project Reunion pivoted into WinAppSDK.
What do you think Qt, VCL, FireMonkey, ImGui, wxWindgets, JUCE make use of?
COM said, "objects are cool and modules are cool so we need an ABI for objects" and specified a vtable. It isn't the only programming model which continued from the same observation: Python did too with its absolutely abysmal leaky PyObject ABI
On OS/2, Smalltalk enjoyed a role similar to what would be .NET and VB on Windows.
Sure you can use delegation with aggregation, and type libraries (nowadays .NET metadata), but still isn't as ergonomic.
Here is an earlier comment of mine with some details - https://news.ycombinator.com/item?id=49142987
[0]: https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2008/n26...
[1]: https://stackoverflow.com/q/12199710
A nice succinct post by Pavel Khaipov; Copy-on-Write in Modern C++: Why the Standard Dropped It - and Why Qt Still Uses It - https://www.linkedin.com/posts/khaipov_copy-on-write-in-mode...
A detailed post by Andrii Nikishaiev; Why Copy-On-Write (COW) removed in C++? - https://www.linkedin.com/pulse/why-copy-on-write-cow-removed...
The std document on Concurrency Modifications to Basic String - https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2008/n25...
Finally, C++11 tightened the specification for std::string viz. contiguous memory, some methods must have O(1) access time, when references/iterators/pointers can/cannot be invalidated etc. which doomed COW.
Program Obfuscation via ABI Debiasing (pdf) - https://www.google.com/goto?url=CAESYgHuR6pNsi8_4x_I2WBy7lmf...
PS: James Coplien in his excellent Advanced C++ Programming Styles and Idioms shows many techniques one of which is actually replacing vtable entries at runtime to mimic features from more dynamical runtime languages (eg. Smalltalk).
In other words, the ABI we rely on today isn't really part of the C+ standard—it's more like the Itnaium C++ ABI or the MSVC C++ ABI.
In the end, I think the ABI stays stable because of community conventions established by compiler vendors.
ABI is more of OS-level thing. Most systems these days follow the SysV ABI, which is largely defined by the hardware manufacturers via the processor-specific supplements (the x86-64 one is here: https://gitlab.com/x86-psABIs/x86-64-ABI). These largely delegate C++-specific conventions to the Itanium C++ ABI (which they likely directly reference), although the ARM ecosystem uses a somewhat different layout for the exception handling tables. Microsoft uses a different ABI for both the underlying C ABI and for the C++ compatibility layer built on top of the C ABI.
One of the issues that crops up is that vendors end up needing to add extensions to the ABI for various reasons, and these extensions tend to end up being incompatible, since they're added before they've had a time to be standardized. 16-bit floats is a particular historical bugbear, as is the C23 _BitInt stuff.
With exception of Swift, D, and bytecode based languages, the ABI is left to the vendors.
For those that think ISO/IEC 9899:2024 PDF has anything related to ABI, the actual ABI used by C compilers, is the OS ABI, if the OS was written in C to start with, and naturally this overlaps quite nicely with UNIX like OSes, and Windows.
It isn't like that on other platforms that decided to either use other systems languages, or expose their OS APIs in a different way, e.g. mainframes, micros, Android, ChromeOS, WebOS,...
I don’t see exceptions there. Apple defines the ABI of Apple’s Swift’s implementation, Walter Bright (or his team) defines that of D, Python defines its ABI, etc.
If you were to write a Swift/D/etc compiler, you’re free to define your own ABI. Disadvantage is that you would give up linking with code compiled by the other compiler (workarounds such as pragmas, C++’s extern "C", are possible)
You're free to do whatever you feel like on your implementation, including not being compliant with the official language standard.
Same applies to C and UNIX/POSIX, OpenGL/Vulkan/OpenCL/NVN/LibGNM(X),...
Thus analysers and language subsetting tools, enforcing language style guides and safer coding practices.
It is complex/complicated/baroque but for the power, flexibility, real-world-usage that it provides it is worth the effort. The mistake that people make is to try and learn it all at the same time which overwhelms them.
Use a good book like Discovering Modern C++: An Intensive Course for Scientists, Engineers, and Programmers by Peter Gottschling and you should have no problem at all.
If you are on e.g. z/OS you would be using ILE, Integrated Language Environment, on ChromeOS JS/WASM, on Android either DEX or JNI,...
Afaik Carbon is at this point the only attempt at a successor language that's still going?
Anyone that can reach out to Rust, Go, Java, C#, whatever, should do that preferably.
There are also some efforts to tame existing C++ with profiles, and replacing UB with erroneous behaviour, and that's about it.
The C++ standard did force gcc to break the ABI of std::string (copy on write was banned - for good reason). They then watched the gcc community work through 10 years of pain to make the transition. They are also well aware that python 3 broke compatibility with Python 2 - and again it resulted in 10 years of pain for the python community to deal with that. With this history there are a lot of experts strongly against any breaking change. It might happen anyway, but only with strong justification and likely an attempt at a migration of some form (what? there are a lot of examples of migration plans that don't work that they are aware of)
Again, it is not because of convention, it is because of painful experience from those who break it.
During the same time the MSVC compiler broke ABI multiple times and the world didn't end.
The fact that breaking ABI was such a shitshow for libstc++ is more related to the way C++/ABI/SOs is/are handled in Linux...
Microsoft never said it outright, but I’ve always assumed the major con that made them switch was that people didn’t buy new Visual Studio versions because upgrading VS required also upgrading all of your binary blob closed-source dependencies.