Suppose you were to throw the whole thing out and write a good replacement (and backwards-compatibility be damned), what would it be like?
Steal the best bits from Windows NT, and improve the existing mechanisms.
Kill signals in their current form. Build a general-purpose notification mechanism consisting of a mutex and a message. Possibly allow a process to have more than one message queue (Windows makes this really, really easy).
All IO, networking and informational signals (SIGWINCH, SIGCHLD etc) then come as messages (these may have to be fixed size, but anything from a few words to a 4k page would do). select, poll etc are replaced by waiting on a mutex. You can put all your worker threads waiting on that mutex. A message arrives. The kernel wakes one waiting thread and gives it the message (via an atomic deque-or-block syscall). You don't have to do any O(n) processing to work out which socket it relates to as the kernel has helpfully put it in the message.
In the deluxe 4k page version, a 1500-byte ethernet frame arrives, is DMA'd into the top half of a page, the kernel inspects it and sets message headers to say where the data is, and hands it directly into the userspace of a waiting process.
The one downside of this is that UNIX pipe programs become slightly more complicated. Rather than just doing "while(read()) write()" you'd have to switch on the type of message recieved and implement your own abnormal exit functionality. This could probably be tidied away for you inside the standard library.
External process control mechanisms would have to be built for killing processes and suspend/resume.
Unix already has a general notification mechanism in the form of poll and select, no need to add a new one. The problem is not all interesting events are not portably delivered via a file descriptor, but that can be more easily extended (as done by lot of unices, including linux) than coming up with a completely new primitive.
But some messages really must be delivered synchronously and can't normally be queued: SIGSEGV, SIGFPE, SIGBUS, etc. There is really no way around interrupts.
BTW mutexes are not for signaling. What you want for signaling in a queue are semaphores, events or condition variables (or even file descriptors, like eventfd).
Well, the brief was to "throw the whole thing out", including select (which is bad) and poll (which is merely adequate).
The machine traps are interesting in that they should only be generated locally - there's no sensible case for injecting SIGSEGV into other processes. Arguably we should learn from Windows "structured exception handling" here. There are two sensible things to do with traps (other than sudden death): hand over to a callback of some kind (which should be told about the state of the stack), or turn into a language-native exception and throw that.
Poll is perfectly fine for the very large majority of unix applications which do not need to scale to tens of thousands of sockets.
The handling over to callback is exactly what is done by unix signals. Converting to exceptions can be implemented on top of signal handlers, but note that even MS stopped mapping by default structured exceptions to language exception a while ago, at least in C++, as unwinding the stack, destroying state and potentially calling destructors is the last thing you want on a segmentation fault or other unexpected events.
The handling over to callback is exactly what is done by unix signals
Not quite, there are quite a lot of restrictions on what you can do in a signal handler. It ought to be possible to design a callback mechanism without those restrictions. And a signal tells you nothing about its origin or what file descriptor / child process etc. it might relate to.
I assume that by restrictions you are talking about the async signal safety; this is inherent on the 'interrupt' nature of signals as they can happen at any point in a program execution, there is really no way around that. It would be of course nice if more functions where async signal safe (especially malloc).
Regarding the lack metadata, I agreed else thread that messages that carry such data ought to be transported via an explicit message queue, not via signals.
The equivalent of SIGSEGV/SIGBUS on Mach is handled in basically the way he describes. On an access violation, it suspends the thread and delivers a message to the registered port. The thing listening on the port (in a different thread or even different process) receives the message, does whatever it needs to do, and then sends a response, after which the original thread is woken back up. From the perspective of the violating thread it was handled synchronously, but the actual implementation was an async message queue.
It is not much better though. If it is another thread it has to work under the same async signal safety rules of a signal handler (the blocked thread might be holding an arbitrary mutex). If it is another process, there isn't a lot it can do.
Now go and read https://news.ycombinator.com/item?id=11864211 . There are those who argue (at some length, q.v.) that "the best bits from Windows NT" are not to have readiness-oriented application designs in the first place.
I was thinking more along the lines of Windows "registered I/O" and Linux "netmap". 'trentnelson' argues for a distinction between readiness and asynchronous. I don't think the distinction needs to exist, and that in an ideal world the disk subsystem would be more like a kind of networking subsystem. If the request can't be satisfied from a RAM buffer, add it to the "outbound requests" list and return to the calling thread. When the corresponding reply eventually turns up from the disk, add it to the input queue of the process.
Steal the best bits from Windows NT, and improve the existing mechanisms.
Kill signals in their current form. Build a general-purpose notification mechanism consisting of a mutex and a message. Possibly allow a process to have more than one message queue (Windows makes this really, really easy).
All IO, networking and informational signals (SIGWINCH, SIGCHLD etc) then come as messages (these may have to be fixed size, but anything from a few words to a 4k page would do). select, poll etc are replaced by waiting on a mutex. You can put all your worker threads waiting on that mutex. A message arrives. The kernel wakes one waiting thread and gives it the message (via an atomic deque-or-block syscall). You don't have to do any O(n) processing to work out which socket it relates to as the kernel has helpfully put it in the message.
In the deluxe 4k page version, a 1500-byte ethernet frame arrives, is DMA'd into the top half of a page, the kernel inspects it and sets message headers to say where the data is, and hands it directly into the userspace of a waiting process.
The one downside of this is that UNIX pipe programs become slightly more complicated. Rather than just doing "while(read()) write()" you'd have to switch on the type of message recieved and implement your own abnormal exit functionality. This could probably be tidied away for you inside the standard library.
External process control mechanisms would have to be built for killing processes and suspend/resume.