This blog forms part of the http://performanceGuru.com website which focuses on performance tuning UNIX-like Operating Systems (including Linux).
Today for whatever reason (Linux kernel crash dump analysis) I was brushing up on my Intel assembler. It was nice to be that close to the iron again and the book that I use (Programming From The Ground Up - An Introduction To Programming Using Linux Assembly Language) started me thinking on the Tube (I live in London, England) about hand optimised code. It was so nice to be writing assembler and be able to use objdump to get back what I had written prior to the generation of the object code and the subsequent topping and tailing of the linker.
It was fun to think about this for a short while on the Tube but eventually I had to admit that compiler writers are smart people and that compiler optimisations are well understood and now into their third or fourth generation. But its nice to dream that Sinclair Spectrums and Commodore 64s are still here and that all that memory has made us bloated, middle-aged and sloppy (instead of that processor performance has increased at four to five times the rate of hard drives, meaning we need relatively more cache than we once did).
Oh well - back to reality...
I have spent quite a bit of time recently looking at Dtrace and Systemtap. The idea is not new, research seems to indicate that the first attempt as this type of probe insertion to allow profiling of code may have appeared in OS2. Both of these tools are really opening the lid on the Operating System, from even simple requirements (who is wrtiting what to disk, the elusive dtop command) through to complex tasks (what causing all those cross calls).
Both allow the horizontal and vertical views that IT always seems to provide. The vertical view is the deterministic profiling option, in which is is possible to follow a single thread of execution, taking sampels at function boundaries, to allow for that thread to be highly optimised. On the horizontal side is the ability to take systemic views which are more akin to statistical profiling.
However there are still a lot of deployed instances that are neither Dtrace or Systemtap capable. For this reason it would seem to be useful to allow for a newer tracing capable O/S to be booted and 'unioned' with an existing Production system to allow for an application to be profiled outside of core Production hours without upgrading the O/S.
For Solaris it would seem to make sense to boot some kind of network avaialbe Solaris 10 instance that maps in the existing root read-only into a zone and then uses this configure the Solaris 10 instance and to provide the local libraries for the application.
For Linux it w0uld seem sensible to do something similar with initrd and PXE. This would allow for an application to be booted using the existing read-only root mounted onto the initrd image.
The two major issues would seem to be versions of the ABI. Solaris should be fairly immune to this as Sun gurantee total backwards compatability and guarantee to fix any issues as bugs. In the RedHat world the big divider would seem to be the threading model, however these kinds of problems can be overcome using LD_ASSUME_KERNEL if necessary. Usually RedHat seem to be very careful with the ABI and things, except for JVMs which seem to play with the stack, etc for efficiency/performance reasons, so for most applications they should work.
Maybe time to do some further research into this. If and when I get time to explore this idea I will post an update.
Have not posted for a long time. Am hoping to be more regular in future. Am currently working on a Perl based tool which will work on Solaris/Linux to show various information from /proc. Am currently looking at possible UI's for the tool. Have been looking a vxWindows bindings for Perl - looks quite good but there is not much in the way of documentation. Might look at TK/Perl - there is even an O'Reilly book on it. GUI program development is quite an education, very different from normal scripting, etc.