I keep moving random bits of code to GitHub by moving a script that creates fish functions for each Mac OS X application. The initial version was presented on this blog along with the fish shell. I added some documentation and fixed a Unicode encoding bug.
It is funny to think that the same year Microsoft introduced Windows 10, Apple introduced the 11th version of Mac OS X, where X was supposed to stand for 10. I just installed it on my laptop, and it was one of the most boring updates I ever did – which is in my opinion a very good thing: I barely noticed that the system was changed, my application seem to still run the same.
The few interface tweaks are nice additions, mostly features ported back from iOS – in particular for e-mail. I liked the new internationalization options, they finally let me set up the computer with the options I want, even if they are a bit crazy.
There are many small tweaks, I love the way terminal now displays password prompts for
sudo, I have to play around with a new
OSC escape sequence introduced in terminal. It lets (I think) the shell communicate the current working directory to the terminal, so new tabs are opened in the same directory.
Yesterday I used Apple’s continuity feature, it was not something I intended, I just received a phone call while using my laptop, and my phone was somewhere else in the flat: the Mac offered me to answer the call. Using a feature, not because you intend to, but because it is the natural thing to do is the sign of a good design. While the sound quality was not perfect (I was asked if I was on the move), it was enough for a call.
I really hope that more applications adopt this model, an applications that triggers notifications on many devices, regardless of the device you are using is very annoying. Skype is pretty bad in this respect: an incoming call will make all devices ring, even after I picked up with one of them. There is a certain irony of having a call disturbed by the noise of the ringing of said call.
It is interesting to see that routing a classical phone call through my local area network is a new feature in 2014. While IP telephony is widely used, it is mostly deployed in corporate network, with a separate network infrastructure to support it, reliability and integration are still pain points. Most chat and video conference system run on proprietary protocols. In that respect, Apple’s approach of building improved services on top of the telephony network instead of trying to replace is interesting: you can see it as a political move to not anger operators, or a way to use a reliable infrastructure for voice.
One thing that used to fascinate me in Ikea shop were the fake computers: plastic shells, with the screen’s content printed out on some adhesive transparency. They never contained a real user interface, more a designer’s vision of a UI, with an extremely high-resolution, you could never see the pixels.
When I installed OS X version 10 on my laptop, it was connected to an external display, i.e a low resolution screen, and my first reaction was, it look horrible. Then I disconnected the displayed and used it as a standalone laptop, and my thoughts were that someone finally implemented this Ikea user interface: the font is Helvetica and the user interface is extremely pure, a lot like the tools I have seen to do UI mocks.
While there still are pixels down the line, they are not relevant anymore, this new UI could have been draw using vector graphics, while this is not the case, just a trickery of multiplying everything by two. Vector logic will come later, maybe. The important thing is that pixels are not important anymore, in a sense this is the same transition as when colour stopped being an issue: pixels were in RGB, no discussion.
Under the hood, I could not find many changes, the
terminal program has become more user-friendly with mouse-scroll and search support, sometimes it scares to see how far this DEC terminal emulator was dragged. I did not have the opportunity to test out the continuity features, i.e. the ability to answer phone calls on the laptop. What I find interesting is that my phone now always appears among the possible WiFi hot-spots.
I have the feeling that Apple is somehow following Intel’s tic/toc approach to release, one of two has many internal changes while the other is about UI and new APIs. Yosemite clearly falls into the later. With its focus on extensions and integration APIs, it was to be expected to have few visible changes, they will come when developer embrace them.
If you have a Mac with a retina display this update is really nice. For the other, the new UI might be a problem. As always, Ars Technica has a long and exhaustive review.
I managed to improve my xkbgrowl program a bit this week-end, the icon on the program sending the event is now converted to a growl icon within the program and not round-tripped via the file-system. Using the file-system has the advantage that I could use existing APIs: X11 knows how to export files in the x-bitmap format, a format that strangely enough, OS X still can read. Safari on OS X is one of the last browsers that supports that strange format (don’t bother checking for buffer overflows, I already did).
Doing the conversion explicitly in the program required some API archeology, I used XLib a bit when I was in the university, but I have largely forgotten what I did then, and I was mostly concerned in pushing pixels out, her the goal is to read some structures provided by the window manager.
I managed to get something that kind of works, which basically mimics the behaviour of
XWriteBitmapFile, resolving the pixels using
XGetPixel. This means I can get the same black/white images I already had, but instead of pushing to a file, I write the data into a buffer which I give the Core Image, which I used to scale the image and serialise it for Growl.
The frustrating part is that
XGetPixel does not return information in
RGB format, instead I get a reference in some colour table. From the documentation, I should call
XQueryColor to resolve the colour into RGB, but when I do so, the whole program crashes with an illegal access. So I’m stuck with OK images for basic programs like
xlogo and broken ones for things link
It is interesting to consider the two graphic APIs I’m trying to connect: their designs are a quarter century apart the first one handles colours as precious resources that are named and allocated and associated with a screen, you cannot simply work with an unattached image buffer. The other is all about buffers and transformations, memory is not the main constraint, but structuring the data-flow in a way that is understandable for the GPU is.
Anyhow, the new version is available in the releases section of github.
Oracle has finally released the version 7 of the Java Runtime Environment for OS X in the same way than the other platforms. This marks the end of a long story: Sun never properly supported Java on the Macintosh – I remember using the version bundled by Metrowerks to work on my Diploma. With OS X, Apple took over the maintenance and integration, and it has recently decided to step down and pass over that responsibility to Oracle.
I don’t think this change will have any serious impact, Java on client machines was dying, and Oracle’s antics have certainly not improved the situation. As far as I know, I only have only run two Java application in the last six months, a configuration utility for my printer, and the software for my tax declaration. I have run more tcl-tk applications in the same period.
During my PhD I wrote some applets (besides the Java Quickdraw library), like the one in the side of this text. I’ll add them to this blog when I have time.
I installed Mac OS X 10.8 on my laptop this week-end, and as far as operating system upgrades go, that one was pretty uneventful. The only problem I encountered was Calendar, iCal’s replacement, getting very confused with my preferences, so I needed to erase them and recreate them. This was trivial, and given the amount of hacking, dog-fooding of internal APIs and general abuse that configurations had received over the years, this was a much needed reset.
Ars Technica, has, as usual, a very long and thorough review. From my perspective, the changes were minor and mostly welcome. Safari is more stable, Address Book has again a usable structure, and there is Reminder application that is equivalent and synced with iOS. RSS feeds are officially dead, which as far as OS X support goes, is no drama, the built-in applications never supported RSS properly. I’m much more annoyed about the death of RSS as open-format as opposed to the social networks.
Under the hood, the main change is certainly gatekeeper the code signing and sandbox system. Generally speaking, desktop applications have way to many permissions, and so being able to control what they do is a good thing, as far as I’m concerned this is just the continuation of having memory protection and pre-emptive multi-tasking. Also being able to set up a Mac in a way where it only runs code from trusted sources is a very much desired feature, in particular for machines used by non savvy persons. I also think it might be a good idea to adapt command-line tools to use the sandbox API and limit their access,
awk should never open a network socket, so why not declare this at startup time?
Lately there have been a few people with Linux backgrounds trying out OS X, so I realized I tended to always give the same tricks and explanations, so I might as well factor them out here. The first thing I would recommend Linux users is to take a deep breath: Mac OS X is a form of Unix, but it is not Linux. It is closer to BSD or even Solaris. Some things are different, get over it. Some things are more difficult or impossible some are easier and some hacks are not needed, like the hacking around needed to have an ssh-agent running.
You will probably want to launch the terminal and get back to a command line interface, the terminal is, from the root disk
Applications ⇒ Utilities ⇒ Terminal. By default OS X comes with a reasonable set of BSD command-line tools, but there is a good chance that your favorite tool is not present, to get it, you will need to do two things: install the developer tools and the darwin port package manager. The developer tools are an optional install on the installation DVD provided by Apple. You can also download them from the Apple web-site, you just need to register. Once the darwin-ports is installed you can go forward the Unix way:
You can install most open-source tools. OS X has an X11 server installed, but it is only started on demand, this is why the
DISPLAY variable contains a weird looking socket. The socket connects to launchd that will start the server on the first connection.
While it is possible to install familiar tools, there are a few Mac OS X only command-line tools that are worth considering:
- This is probably the most handy Mac OS X command-line tool, it basically opens documents and launches GUI applications. You can use it whenever you want to emulate a click on a Finder Icon (the file-system explorer). So if you want to open the current directory
open .. The
opencommand also understands URL.
- pbcopy, pbpaste
- Those two command put the content of
stdininto the clipboard and dump the content of it to
stdout. Basically they have the same functionality than
- Dumps the system profile, this is useful to have a view of what hardware the operating system has detected.
- mdfind, mdls
- Those commands give access to the Spotlight meta-data. They are useful for searching for files and inspecting their meta-data respectively.
- hdiutil, diskutil
- Those are the tools to manipulate disk images and actual disks. They expose the functionality of the GUI tool Disk Utility.
- Well behaved Mac OS X applications store their preferences in standard property files. The files are in
~/Library/Preferences, but you can just manipulate them using the
- Outputs the content of
stdinto the speech synthesizer. Useful for long running scripts.
- This command lets you execute Applescript commands from the command line. This means that you can control GUI programs like Word or Photoshop from the a shell script. The langage can be quite complex, but you can do quite neat things. For instance this small script makes an application quit cleanly.
#!/bin/tcsh set osa = 'Tell Application "'$1'" to Quit' osascript -e "$osa"
All those commands have man pages (and none of this info non-sense). The Mac OS X hints web site is also a good source of information on ways to hack the system. One trick I used regularly is this script to create bash aliases for all GUI applications.
One of the fascinating things about the demo-scene is the amount of things that can fit in 4K of code. On the other hand, when one compiles a simple program under a Unix system, the resulting binary is usually very large for its functionality. Simple programs like /true use more than 12 Kilobytes. What the hell is the reason? Under Mac OS X, we need to look at the Mach-0 binary file format for an answer.
A handy tool to look into binaries is
otool. What can it tell us? First let us look into the the header:
otool -h /usr/bin/true
/usr/bin/true: Mach header magic cputype cpusubtype caps filetype ncmds sizeofcmds flags 0xfeedface 7 3 0x00 2 13 908 0x00000085
A Mach-0 file is a sequence of commands that tells the operating system how to set-up a process for starting. The file has first a header, then a sequence of commands that can reference positions later in the file.
Here we see that setting up the
true requires 13 commands that take up 908 bytes. Let’s see what those commands are.
otool -l /usr/bin/true /usr/bin/true: Load command 0 cmd LC_SEGMENT cmdsize 56 segname __PAGEZERO vmaddr 0x00000000 vmsize 0x00001000 fileoff 0 filesize 0 maxprot 0x00000000 initprot 0x00000000 nsects 0 flags 0x0 Load command 1 cmd LC_SEGMENT cmdsize 124 segname __TEXT vmaddr 0x00001000 vmsize 0x00001000 fileoff 0 filesize 4096 maxprot 0x00000007 initprot 0x00000005 nsects 1 flags 0x0 Section sectname __text segname __TEXT addr 0x00001f98 size 0x00000066 offset 3992 align 2^2 (4) reloff 0 nreloc 0 flags 0x80000400 reserved1 0 reserved2 0 Load command 2 cmd LC_SEGMENT cmdsize 192 segname __DATA vmaddr 0x00002000 vmsize 0x00001000 fileoff 4096 filesize 4096 maxprot 0x00000007 initprot 0x00000003 nsects 2 flags 0x0 Section sectname __data segname __DATA addr 0x00002000 size 0x00000014 offset 4096 align 2^2 (4) reloff 0 nreloc 0 flags 0x00000000 reserved1 0 reserved2 0 Section sectname __dyld segname __DATA addr 0x00002014 size 0x0000001c offset 4116 align 2^2 (4) reloff 0 nreloc 0 flags 0x00000000 reserved1 0 reserved2 0 Load command 3 cmd LC_SEGMENT cmdsize 124 segname __IMPORT vmaddr 0x00003000 vmsize 0x00001000 fileoff 8192 filesize 4096 maxprot 0x00000007 initprot 0x00000007 nsects 1 flags 0x0 Section sectname __jump_table segname __IMPORT addr 0x00003000 size 0x00000005 offset 8192 align 2^6 (64) reloff 0 nreloc 0 flags 0x04000008 reserved1 0 (index into indirect symbol table) reserved2 5 (size of stubs) Load command 4 cmd LC_SEGMENT cmdsize 56 segname __LINKEDIT vmaddr 0x00004000 vmsize 0x00002000 fileoff 12288 filesize 5296 maxprot 0x00000007 initprot 0x00000001 nsects 0 flags 0x0 Load command 5 cmd LC_SYMTAB cmdsize 24 symoff 12288 nsyms 2 stroff 12316 strsize 32 Load command 6 cmd LC_DYSYMTAB cmdsize 80 ilocalsym 0 nlocalsym 0 iextdefsym 0 nextdefsym 1 iundefsym 1 nundefsym 1 tocoff 0 ntoc 0 modtaboff 0 nmodtab 0 extrefsymoff 0 nextrefsyms 0 indirectsymoff 12312 nindirectsyms 1 extreloff 0 nextrel 0 locreloff 0 nlocrel 0 Load command 7 cmd LC_LOAD_DYLINKER cmdsize 28 name /usr/lib/dyld (offset 12) Load command 8 cmd LC_UUID cmdsize 24 uuid 0x7d 0xf6 0x04 0x33 0x2c 0xb5 0xc7 0x5c 0x3a 0x53 0xe1 0xd2 0x4b 0x5b 0xa3 0xac Load command 9 cmd LC_UNIXTHREAD cmdsize 80 flavor i386_THREAD_STATE count i386_THREAD_STATE_COUNT eax 0x00000000 ebx 0x00000000 ecx 0x00000000 edx 0x00000000 edi 0x00000000 esi 0x00000000 ebp 0x00000000 esp 0x00000000 ss 0x00000000 eflags 0x00000000 eip 0x00001f98 cs 0x00000000 ds 0x00000000 es 0x00000000 fs 0x00000000 gs 0x00000000 Load command 10 cmd LC_LOAD_DYLIB cmdsize 52 name /usr/lib/libgcc_s.1.dylib (offset 24) time stamp 2 Thu Jan 1 01:00:02 1970 current version 1.0.0 compatibility version 1.0.0 Load command 11 cmd LC_LOAD_DYLIB cmdsize 52 name /usr/lib/libSystem.B.dylib (offset 24) time stamp 2 Thu Jan 1 01:00:02 1970 current version 111.0.0 compatibility version 1.0.0 Load command 12 cmd LC_CODE_SIGNATURE cmdsize 16 dataoff 12352 datasize 5232
So what do those commands do?
- The first command (0) sets up a memory segment called
__PAGEZEROat address zero, with special permission. This is basically a facility that makes sures that NULL pointers result in an error.
- The second command sets up the __TEXT segment with the read only binary code. The command tells the system which part of the binary file will be mapped at which address.
- The next command sets up the __DATA segment, this will contain the program’s initial writable memory. As everything needs to be aligned on memory page boundaries, this segment’s representation has to be stored on a different page in the file, i.e. start at offset 4096 even if there was space left before.
- The next command sets up the __IMPORT segment which contains the data structure for importing symbols from shared libraries. As this will be a different page it has to be on another page, we are now at offset 8192.
- The next command sets up the symbol table, this one does not need any representation in the file (but it can reference names in the __IMPORT section).
- This command specifies the dynamic symbol table, again no representation in the file.
- This command specifies what dynamic linker is used, here it is
- This command specifies a unique identifier for the binary.
- This command specifies the register state of the initial thread. Most of the registers are set to zero, except the instruction register (eip) which contains the entry point of the program.
- This command specifies a library to load
/usr/lib/libgcc_s.1.dylib, this is the supporting library for gcc generated code.
- This command specifies another library,
/usr/lib/libSystem.B.dylib, this is the giant framework that contains most of the Unix libraries of OS X.
- The last command specifies the signature for the code
The conclusion here is that the binary file is mostly filled with zeros so that the various segments fit within memory pages. I wrote a small tool that measures how much space is really used, and in the case of true, only 1151 bytes out of 17584 are used, that is 6.55%. I’ll talk about that tool another time…