May 17, 2007
Look, Ma. No framework!
When I first sat down to set up the project files for Sandvox, I knew that we wanted to support plugins. These are little bundles of Cocoa code and related resources that can be loaded into an application at runtime to extend its funcationality. A lot of Cocoa apps work this way. Watson did. So do Mail, iPhoto, iDVD, even Calculator. If an app keeps its plugins in the standard location (Contents/PlugIns) you can even browse them and activate/deactivate, add or remove them via Finder.
If, in Finder, you do Get Info on Sandvox you'll see the list of plugins that are included in the .app.
Though you can activate and deactive them using Finder, I don't recommend doing it, for any app. Apple seems to have implemented half of a good idea here, but with no documentation or expectation as to how it should be used. Activating/deactivating plugins this way may have unfortunate results. So don't do it. But it's still an interesting way to browse and see if an app supports plugins, at least in the standard way.
Nevertheless, what's important is that plugins allow you to modularize and build out an application's functionality over time, including after the bulk of your app has already shipped. Not only can you extend your application with new plugins, but so can third parties. That's a big bonus and well worth supporting in your application if it makes sense.
The typical way that you encourage plugin development for your app is to move the general code required to support a plugin to its own framework that you then bundle inside your .app. Another developer comes along, learns (via SDK documentation, if they're lucky) that your app supports plugins, and that the way to do that is by linking to one or more frameworks within your app's main bundle.
Sandvox works this way. In fact, Sandvox supports six different types of plugins: Pages, Pagelets, Elements, DataSources, Indexes, and Designs. In all shipping versions of Sandvox up to the current, 1.2, we've made available a framework called KTComponents that you need to link your plugin against so it knows how to load itself into Sandvox, talk to Core Data, etc.
Wanting to be as open with our code as is reasonably possible, we kept moving more and more functionality from Sandvox over to KTComponents so that plugins could use this code, too. This included things like HTML parsing, UI classes, value transformers, and a bunch of different categories extending not only Foundation and AppKit, but Core Data, Core Image, and WebKit. Over a hundred classes or categories in all. A lot of code.
And, frankly, it's been a big pain maintaining it. Having all this code in a separate framework slows down the build. Makes the build target more complicated. Makes it harder to figure out where to set breakpoints. Makes it harder to locate resources. Makes maintaining localized strings more work that it should be. And did I mention that it really slows down the build?
Well, no more. When the next major update to Sandvox ships, KTComponents will be no more. All the code is going into the app. And plugins will work just fine. In fact, getting new plugins to build and link will be easier than I ever thought possible.
How can this be?
About a year and a half ago Wil Shipley posted a piece called Frameworks are Teh Suck, Err wherein he makes many fine points about where framworks came from, what they're good for, what their limitations are, and why you shouldn't use them.
At the time I thought "Well, that's nice, Wil. But we have to support plugins and we need a framework for that." Dan had posted in the comments to Wil's piece asking about this, but no one offered up a solution. I didn't think much about it at the time; just resigned myself to the annoying code split and build headaches.
About a year later, Wincent Colaiuta posted some thoughts about Cocoa development and the things he wished he'd done a little differently, had he only known, including where and when to use frameworks. Included in his post is a little piece of linker trivia that I had never heard of before. A flag for ld called bundle_loader.
Wincent didn't include any links to any information about bundle_loader and it only appears once in Apple's documentation, under Dead Code Stripping. I didn't think much about it at the time: just filed it away until I had some time to explore. This week, I explored...
Here's what 'man ld' has to say about bundle_loader:
-bundle_loader executable (32-bit only)
This specifies the executable that will be loading the bundle
output file being linked. Undefined symbols from the bundle are
checked against the specified executable like it was one of the
dynamic libraries the bundle was linked with.
I had no idea. This is basically how you get a plugin to link against an app without the linker going belly up. It works and it works well. (The 32-bit only comment is a slight concern. I don't think it will be an issue for the foreseeable future for Sandvox. I intend to confirm this at WWDC.)
If you've made it this far, rather than bore you with all the things I tried to make the linker happy before I stumbled onto bundle_loader, let me just share with you how we're going to be building plugins from here on out.
Rather that having to know about and locate KTComponents in the Sandvox appwrapper, somehow adding that to your plugin project, getting your framework search paths right, worrying about framework version numbers, if you're so inclined, and all that other junk, you now just add these two settings to your plugin's target's build configurations.
USER_HEADER_SEARCH_PATHS = /path/to/Sandvox.app/Contents/Headers
BUNDLE_LOADER = /path/to/Sandvox.app/Contents/MacOS/Sandvox
That's it! Really, that's it. That's all you have to do. Your plugin doesn't have to know anything about Sandvox except those two settings. Unbelievable. What was I doing messing around with frameworks for all these years?
You can put these in the Build tab of your target's inspector or you can put them into xcconfig files.
Suddenly, now, life just got a whole lot simpler. Plugins are trivial to get up and running. And we now have all the Sandvox code under one roof. (ConnectionKit, iMediaBrowser, and WebKit notwithstanding.) Suddently refactoring Sandvox becomes doable, since all the code is in the same project now. Very happy news. And, oh yeah, did I mention builds are a lot faster?
One last word of advice: to make this work, you do have to pay attention to what symbols you're exporting. Typically when you make an install build, the executable is stripped of many of its symbols: symbols that plugins are going to need to link against. Basically you can either leave your app unstripped (though still strip debugging symbols) or pass flags to strip that are more akin to those used with frameworks than with apps. E.g.,
STRIPFLAGS="-S"
And if you have things marked "extern", like string constants and functions that you want accessible across files, you need to tell gcc not to make those symbols private or plugins won't link for that reason as well. E.g.
GCC_SYMBOLS_PRIVATE_EXTERN=NO
So that's the big secret. No more splitting projects into app+framwork(s). I wish I had known about this years ago.
bundle_loader is the way to go.
Posted by ttalbot at 3:06 PM | Comments (2)
May 16, 2007
Fail Faster
The default Cocoa Application project in Xcode includes a target with a set of build phases. Here's what it looks like:
Notice that Copy Bundle Resources happens before Compile Sources and Link Binary With Libraries. This means that every time you build, before the compiler or the linker see anything, Xcode spends time copying things like nib files and images into the built product.
If you have a large app, this list of resources to copy can be quite long. Everytime you make a source change and hit command-Y to compile and relaunch your app, it has to run through Copy Bundle Resources again, and then it tries to compile. That's fine. It needs to do that. What's the big deal?
The big deal is that if you're writing a bunch of new code or manually refactoring some older code and you have syntax errors, say, a missing semicolon, a typo in a class name, or a missing #import, the compiler is going to complain and halt your build. If you forget to add a reference to a framework or a library, the linker will complain and halt your build. But it's going to halt your build after copying bundle resources, things that the compiler and linker pretty much don't care about.
In Sandvox's case, there are nearly 200 different resources to copy during a build, and that's not even counting the plugins and frameworks that ship with the app. Doing this over and over again during the course of a day adds up. That's time we can reclaim.
Whenever I make a new project now, I open up the target(s) and move Copy Bundle Resources below Link Binary With Libraries.
This makes sure that the first thing Xcode does is build my code. If it builds, then it can go on and flesh out the rest of the .app. If it's not going to build, I want to know right away so I can spot the problem, fix it, and get back to work.
So I say, move that build phase!, and save yourself some time.
Posted by ttalbot at 4:25 PM | Comments (2) | TrackBack
September 25, 2006
Thanks Mickey
I originally wrote this at the very end of last year, but apparently never published it. So, old topic. But I liked the post anyway, so here it is...
No, this isn't about Disney. It's about Michael Sprague, my high school computer science teacher. I no longer see him on the school's website. He'd only be about 50 now. A little young to retire. But maybe not. I hope he found something better to do than teach teenagers about computers, but he was pretty great at that. So actually I hope he's still doing it.
What brought "Mr. Sprague" to mind was Joel's latest [ed. not so latest anymore] essay, The Perils of JavaSchools, and Dori's followup, You think you're old? recounting her computer science education back in the day, as the kids say.
Dori nicely summarizes Joel's argument: "[In] the old days, programmers went to college, learned C, and got a good understanding of what was going on deep down internally. Nowadays, these newfangled programmers are only learning Java, and consequently, they don't know anything about recursion or pointers. Schools should teach Scheme and C, and the students will be the better for it."
And that got me thinking. Pointers and recursion. Yeah, he's right. Sprague had us doing those in PDP-11/70 assembler and Pascal (for the AP test the following year) and some pseudo-language of his own invention when I was a junior in high school.
I mean, like, for months it was dereference this, parse-this-expression-into-a-tree that. And after a while our heads did explode. But we felt better for it. And, you know what?, the following year the small group of us that could put our heads back together again won the ACSL state championship really without even worrying about it. (Ok, so we placed, I think, 9th nationally but that had more to do with our hastily arranged trip to Amish country than anything else.)
This was a good thing, too. When I went to college, a really, really long time ago, at least in Internet time, my alma mater didn't even offer a formal computer science undergraduate education. (It was considered "too practical". Apparently that's now changed.) So I went to law school...
Now I don't bring this up to say "see, I was doing this in high school, look at me", I bring this up to say a sincere thank you to whomever at LHS decided they could start a computer science curriculum around an 8 year old 16-bit minicomputer and to give a tremendously big shout out to Mickey. You guys saved my butt.
Posted by ttalbot at 10:17 PM
September 2, 2005
AppleBugFriday: Xcode updates wipe out documentation bookmarks
Since it's Report-An-Apple-Bug-Friday, I thought I'd pick one that hits me where I live everyday. Xcode's documentation viewer. Even though I've had more than a passing familiarity with the AppKit for, um, longer than I might care to admit, Tiger brings with it a fairly hefty chunk of new API. (Not that I'm complaining!)
As I've gotten older, I've come to realize that the available mental stack space I have is finite. I can't remember every bit of new API that I come across. Keeping documentation handy is, for me, a very good thing. Xcode helps with this by allowing you to bookmark class documentation so you can easily find it again. (Find > Add to Bookmarks) In practice this works fairly well and let's me, say, flip between related Core Data classes while I'm working on a problem.
Oddly working against this is the increased pace at which Apple is releasing updates both to Xcode and OS X API documentation. (Again, not that I'm complaining! Monthly documentation updates delivered over the web are both new for Apple and a true blessing.) But, and you knew there had to be a but, every time I install either a documentation update, or an update to Xcode itself, I lose all my bookmarks! Really, really annoying.
So that's my Friday bug. It drives me crazy. Mostly because it happens without warning and with no chance to recover.
Following tradition (?) here's the geeky Radar URL: 4242754. I see that's Dan's now posting the text of his submitted bugs. That's a really good idea. I'll append mine to the end of this entry. One of my ideas from the conversation we had that started AppleBugFriday was to post bugs in the hope of getting other people to report the same bug in order to help adjust Apple's priority queue. If someone posts a bug that's affecting you, too, file your own report. Maybe it will help the bug get fixed quicker, or at all. If you have an ADC account, report a bug. If you don't have an ADC account, report a bug.
Mike Zornek also came up with a clever way to raise visibility: tag your entry on del.icio.us with applebugfriday. Make it so.
Here's the text part of my bug:
Summary: Installing Xcode documentation updates removes class documentation bookmarks.
Steps to Reproduce:
1. Bookmark class documentation in Xcode 2.x via Find > Add to Bookmarks.
2. Install a documentation update.
3. Observe that previously added bookmarks are no longer shown in documentation viewer.
Expected Results:
Xcode should maintain bookmarks even after updating documentation. If Xcode really can't find the referenced documentation, the bookmark should not be lost: the user should be given an opportunity to (re)locate the documentation.
Actual Results:
Bookmarks are reset to "standard set" as if I'd never used Xcode.
Regression:
I have experienced this problem with every version of Xcode that has allowed creation of documentation bookmarks.
Notes:
Installing either new versions of Xcode or documentation updates wipes out bookmarks.
Posted by ttalbot at 8:42 AM
July 8, 2005
Diagrams that break
One of the things I've been working on since WWDC is streamlining Sandvox's Core Data model. (Yes, that's part of it there, on the left. It'll likely be public when we publish the plugin spec, so I don't think I'm giving away too much.)
Even though, many months ago, I came up with some really kick-ass model merging code that grabbed partial models from a bunch of different plugins and hooked up all the attributes, relationships, fetch specs, and parent/child relationships in just the right way, it turned out to be absolutely the wrong approach since, as soon as one thing varies in the model – oh, say, the presence or absence of a plugin – you've effectively broken your file format, requiring a lot of gnashing of teeth and grindage of CPU while you migrate from whatever your model used to be to whatever your model is now.
Thunderstruck to learn that I'd have to toss all (twelve lines of) my brilliant code, I decided to just suck it up and simplify, simplify, simplify. (Always, always, always a good thing to do anyway.)
So here's the good news: writing a plugin for Sandvox now requires virtually zero knowledge of Core Data: we just do it all for you behind the scenes using one simple, easy-to-maintain (and upgrade) model. You just create a delegate that implements and customizes what you need to. Just like it should be. Thank you Cocoa.
And here's the better news, at least for me: the modeling tool in Xcode, and Core Data itself, actually made this streamlining really, really easy. How? Visual development of (compiled) models, with copy/paste, and datastores that validate against the model at Save. In other words, Diagrams that break.
As I mentioned somewhat off-handedly, as soon as you start seeing Core Data as fundamental to how you use Cocoa, you stop worrying about a whole class of problems. Not only can you whip up a pretty powerful object model by clicking and dragging (and, well, thinking), the system actually tells you if you get it wrong.
Actually, it does this in two ways: first it compiles your model on the front-end, into a .mom. If you've done something really stupid, your model won't compile. You have to fix it in the modeler before you even get to the code. And then on the back-end, when you go to save a document, Core Data actually verifies that the data you want to store matches the requirements you set out in the model. Say hello, one would hope, to referential integrity. That's not to say you can't put bad data in, but at least you're not leaving good data out. And that, I think, is a real step forward for fast, fast, fast Cocoa development.
On a related note: Apple just updated the developer documentation. Be sure you've got the latest. Xcode Preferences > Documentation > Check Now.
Posted by ttalbot at 7:57 PM | Comments (3)