I’m finally recovered after a really exhausting week that included SocialDevCamp and the wild ride of Twitter Vote Report.

SocialDevCampEast2 went off without a hitch on Saturday at University of Baltimore.  Once again, some of the best and brightest developers, entrepreneurs and social media gurus gathered to trade ideas and talk about the future of the web.

One thing we try to do at SocialDevCamp is vote on the sessions, to make sure they are things that people really want to hear about, or at least size the discussions to the right rooms.  We ran 5 rooms all day in 5 sessions plus lunch, for a total of 25 sessions! Check out the wiki to see the sessions that were held.

Personally, I enjoyed the conversation on location technology, and why location-based social networks have yet to reach critical mass.  Most folks felt that there was a technological barrier — it’s just too hard to continuously update your location with current device and battery constraints — and others questioned what incentives people have to update their locations.  We decided that those incentives probably needed to be tuned in order to see a successful location-based service emerge, and that there may also be benefit for people sharing location-related information anonymously.  Great talk, and I’m still thinking about what incentives might make LBS actually work.

We did a session on Twitter Vote Report, which was awesome because we were actually able to recruit some members of the crowd to do some work on the project!  Bryan Liles and John Trupiano contributed some great work to the codebase, some while sitting in the session!  We talked about the overall architecture of the project, and the fact that it was put together in just two short weeks of coding!

There was a good conversation about iPhone development, introducing people to the platform and answering questions about the platform.  Many seemed to be glad to get a feel for Cocoa and I wouldn’t be surprised if several of the folks there end up working on the platform!

Alex Hillman of Philadelphia’s Indy Hall helped to lead a discussion on co-working in Baltimore, and by the end of the session, we had actually launched co-working in Baltimore, with a mailing list and a set of great ideas for taking things forward.  Yesterday, we held our first “official” co-working meetup at Bluehouse in Baltimore; I’ll write more about the co-working initiative separately.

Because I wasn’t in the other sessions, I can’t say what all was said in them, but I heard good things about the conversations on data portability, source code management with Git, and crowdsourcing. If you were in one of the sessions, feel free to leave some comments here or links to your own blog!

Ann Bernard helped put together an awesome party for SocialDevCamp at Metro Gallery with great food from Tapas Teatro and an open bar.  And live music from Natasha El-Sergany, KADMAN, and Ra-Ra-Rasputin… A great way to end the day, and I can say that by the time it was all over, I had talked to a few hundred people and was completely exhausted!

This morning, Mike Subelsky, a friend and one of the organizers of the recent and fabulous Ignite Baltimore said via email, “It is not an exaggeration to say that SDCE has totally changed my life,” referring to the first SocialDevCamp held in May. Not to sound self-congratulatory, but the same is true for me.

SocialDevCamp is one of a few things sparking a renaissance here in the Baltimore/Washington area, giving rise to events like Ignite and to movements like co-working.  With the social media tools available now, this sort of thing is finally possible to do, and it’s hugely gratifying to see it happening!

See you next spring for SocialDevCampEast3!

Being busy seems to always come in spurts for me… just when it looks like I’ve got too much to do already, something cool turns up and takes things to whole new level of busy.

That would be this week. SocialDevCamp East, the barcamp-style unconference that I started with some friends last spring is back tomorrow, and that’s certainly required some coordination and planning.  That would have been plenty.  We have over 200 RSVP’s now (between the Wiki and Facebook) and we expect a truly incredible day of networking and learning.  See you tomorrow!

The other big news of the last two weeks has been the TwitterVoteReport project, for which I’ve been acting as defacto CTO since about October 18th.  This is a great project, a great cause, and an awesome idea.  The data we collect will be an archival quality primary source document for future generations to study the evolution of the election process.

We have five distinct data sources coming in about people’s experience at their polling places: Twitter, Telephone, Direct SMS, and Apps for Android and iPhone.  These are all normalized and aggregated into a single database and reviewed by humans for maximum accuracy.  The data will then be made available in real time to anyone who wants it — from the media to watchdog groups to mapmakers — to help the world understand and monitor the 2008 US elections.

Putting this project together, with all these diverse inputs, has been a monumental task and a real demonstration of what’s possible when people decide to work together.  We had over 600 phone channels donated.  We were able to think up, code, and submit an iPhone app in just 3 days.  We’ve received press coverage far and wide from sources as diverse as TechCrunch and Fox News.  Not bad for a few days’ work.

There’s plenty more to do still (between now and Monday), and I’m busy all day tomorrow at SocialDevCamp.  We’ll do a session there on TwitterVoteReport and what we’re up to… we still need more help from people good with maps!

I’ll post more here as things evolve, and a recap next week, but remember, nothing’s impossible when caring people dedicate themselves to a common endeavor.

Meantime, check out:

• http://twittervotereport.com
• http://votereport.pbwiki.com
• http://socialdevcamp.net

The self-organizing tech community event has finally come of age here in Baltimore.  Here’s three events you can’t miss.

• Ignite Baltimore, October 16, 2008
• SocialDevCamp East, November 1, 2008
• TwinTech III, January 22, 2009

My new company, Roundhouse Technologies, is a sponsor of all three, and I’m speaking at Ignite Baltimore and am event co-chair for SocialDevCamp.  Each of these events is an example of the kind of self-organizing community events that I believe will shape the next wave of tech on the east coast and which I believe will give rise to the next great wave of innovation. And this time, that innovation is going to happen in places besides the towns along 101 and Interstate 280.

I’ve not talked a lot about Roundhouse yet publicly, but we’re methodically building things up, and we’ll have more to say soon. Stephen Muirhead and I are heading up the company. Stephen is an experienced executive and entrepreneur, and among other distinctions is the former president of Microprose Software, maker of the Sid Meier Civilization games, (ironically now owned by Atari, with which I had a long association, though under a previous incarnation).

So, anyway, lots of stuff is happening.  Ignite Baltimore should be amazing.  If you have not RSVP’d yet, please do so now to be sure you can get in.  The space is limited.  SocialDevCamp East was heralded as one of the top tech events on the east coast, and we’re expecting another amazing day on November 1.  And if Twin Tech II (held a couple of weeks ago in DC) is any indication of the scale and energy we can expect at Twin Tech III, we’re in for a heck of an event.

Tech is very much alive and well in DC, Baltimore, Philly, and New York.  Watch it unfold in the coming months and years!

An hour or so ago I launched Twittervision Election View, allowing viewers to see posts to Twitter about the 2008 election in their original geographic context.

Twitter launched something similar this morning, and the idea to do a political view of Twittervision has been around for a while, so it seemed natural to try to do this now and especially in advance of tonight’s debate.

We have some enhancements planned, and right now the site is getting a ton of traffic as people discover it… we should be able to put some more server capacity on it which should keep things steady.

Let me know what you think!

In May, several of us put together SocialDevCamp East at University of Baltimore.  It was an incredible day, filled with deep technical content as well as excellent discussions of business strategy and the very real challenges that east coast companies face breaking into the consumer-facing Internet space.

The day was universally hailed as a success, and maybe even a little bit of a breakthrough: for the first time, the “Amtrak corridor” tech communities had come together to face the challenges of the future together as a unified ecosystem, not just as individuals.  The result was a phenomenal mixing of technical, business, and artistic topics and in my mind, was a glimpse of the future.

So, we’re back at it on November 1st, 2008.  You can sign up on the barcamp wiki or on Facebook.

We’re looking forward to another great event, and another awesome afterparty at Brewer’s Art.  Go ahead and start your juices flowing for what sessions you would like to see, and post them to the Wiki.

We’re also looking for sponsors for both the event and the after party.  To find out more, please contact us.

See you in November in Baltimore!

When we first launched Twittervision in early 2007, Twitter was still a pretty small community of users (around 200,000) and only the press and the digerati were paying much attention to it.

Today, with just over 1M users, Twitter is still pretty small by Internet standards, but a lot of people are paying attention to it.

Our API was designed to allow individual users to use the Twittervision location features. A lot of people are using it. We also had a fair number of people who were using our API as an alternative to the Twitter API and trying to harvest vast amount of data using our free API.

Sadly, this was restricting service to others, so we are making some changes to the API that make this kind of use no longer possible. Those of you using the API for your individual projects or in support of client-side apps will see no changes for now — keep doing what you’re doing.

We do sometimes engage in licensing agreements, however, so if you are interested in licensing our data, please contact me at dave at twittervision.com.

Judging by the fact that there are several posts about this topic out in the wild, and that I have come across a need for it more than once, I thought it would be helpful to wrap up this functionality into a plugin and put it out into the world. Give a warm welcome to ActsAsRenderer!

Before you go off on a tirade about the evils of violating MVC, let me first say I know the arguments and I agree with you. However, in a world of complex systems where not everything is done via full-stack HTTP, there are legitimate reasons to output data directly from models, and ActsAsRenderer helps you do it.

With ActsAsRenderer, you get four cool new functions.

For your model class, you get render_file and render_string. For your instances, you get render_to_file and render_to_string.

Probably the most common (and legitimate) use of this kind of functionality is for rendering data out of a Rails script (say with script/runner). Since that environment is not a full-stack HTTP view of the world, it’s a real pain to render any kind of structured output. Not anymore! With acts_as_renderer in your model, you can render your views and give your model the voice it’s been lacking!

I’ve had this need come up several times. Most recently, I built a server configuration management system using Rails. While it is nice to preview the rendered configuration files using Rails-over-HTTP, it is also essential to be able to write those same configuration files out to the filesystem. In another case, I had a background DRb process that needed to be able to render templated output to the filesystem. I had to go build a mock-controller and do some pretty unsavory things; all of that would have been obviated with acts_as_renderer.

Now, I can simply say:

class Server < ActiveRecord::Base  acts_as_renderer

  def build_configuration    CLIENT_CONFIG_FILES.each do |f|      render_to_file("configs/#{f}", "#{config_dir}/#{f}.conf")    end  endend

The render_to_file function renders the templates located in configs (under app/views by default) and writes them to the files specified in the config_dir; it’s also smart enough to know that render_to_file is being called from a ‘server’ instance and sets @server accordingly. So my templates in configs are simply:

; Configuration Snippet for Server <%=@server.description%>

<%= render :partial => 'configs/queue', :collection => @server.queues %>

Please do think before using this plugin. It can be used for some seriously evil violations of good MVC design practice, and you are responsible for your own actions. However, this can also be used to make your existing designs *much* more robust and elegant, and I encourage you to use it where that is true.

It’s ready to drop in. Everything is there, including tests. Enjoy!

NOTE: Version 1.0 only supported Rails 2.0; I just added version 1.01 which will work with either Rails 1.2.x or 2.0.x. Please feel free to ping me with any questions.

acts_as_renderer at RubyForge

With the recent release of Rails 2.0, many of us are reviewing our approaches to common problems. Many new features have been added to Rails, and some old tricks are either no longer necessary or no longer work.

I am developing a project with Rails 2.0 and am getting close to putting it into production. A recurring issue for today’s web developers is that of asset packaging, or the combination of multiple site assets into a single file. Specifically, we’re talking about Javascript and CSS.

A given “Web 2.0″ (a term I wish had recently been found dead in a cramped apartment in Brooklyn) site might have a half dozen Javascript or CSS files to deliver to a user, and web browsers are not all that efficient at retrieving them. Each one requires a separate TCP connection to the server, and many browsers are only capable of getting two of these files concurrently. This means delays for your users.

In Rails 2.0 (and previously in Edge Rails), it’s possible to combine multiple Javascript and CSS files using the javascript_include_tag and stylesheet_link_tag functions in your html.erb files; simply add :cache => true to the parameters like this:

<%= javascript_include_tag 'prototype', 'effects', :cache => true %><%= stylesheet_link_tag 'main', 'shop', 'form', :cache => true %>

With :cache => true and when running in your production environment, Rails will automatically combine your Javascript and CSS assets into single files (all.js and all.css, respectively) and significantly reduce your site’s load time.

However, this really only solves part of the problem. A common technique used to further improve site performance is to compress Javascript and CSS by removing unnecessary whitespace and comments. I am not sure why this wasn’t included as part of Rails’ built-in caching features, but it seemed to me it should be easy to add.

Turns out I was mostly right. Google “javascript minimization” (or minification) and you’ll see it’s a pretty hot topic. The Asset Packager plugin from Scott Becker does this, as well as CSS compression, but is targeted at Rails 1.x and doesn’t really make sense in the face of Rails 2.0.

So I set out to solve this problem in an elegant way for Rails 2.0. Asset Packager uses a Ruby script called jsmin.rb by Uladzislau Latynski which is based on jsmin.c by Douglas Crockford. The thing is, jsmin.rb is not a class or library, but rather a standalone executable that operates on stdin and stdout. Asset Pacakger actually forks a ruby shell process to do its Javascript minimization, and this seemed like folly if it could be done internal to Rails.

Accordingly, I modified jsmin.rb to operate as a singleton class and with a class method you could pass Javascript data to. Then it was simply a matter of monkey patching this function into ActionView::Helpers::AssetTagHelper, home of javascript_include_tag and stylesheet_link_tag.

I also wanted to add in CSS compression, which turned out to be easy. The javascript_include_tag and stylesheet_link_tag functions both use the same underlying functions to package their assets, so it was a simple case of replacing them with equivalents that do compression appropriately, based on whether we are dealing with CSS or JS.

config/initializers/javascript_minimization.rb:

module ActionView  module Helpers    module AssetTagHelper      require 'jsminlib'

      def compress_css(source)        source.gsub!(/\s+/, " ")           # collapse space        source.gsub!(/\/\*(.*?)\*\/ /, "") # remove comments        source.gsub!(/\} /, "}\n")         # add line breaks        source.gsub!(/\n$/, "")            # remove last break        source.gsub!(/ \{ /, " {")         # trim inside brackets        source.gsub!(/; \}/, "}")          # trim inside brackets      end

      def get_file_contents(filename)        contents = File.read(filename)        if filename =~ /\.js$/          JSMin.minimize(contents)        elsif filename =~ /\.css$/          compress_css(contents)        end      end

      def join_asset_file_contents(paths)        paths.collect { |path|          get_file_contents(File.join(ASSETS_DIR, path.split("?").first)) }.join("\n\n")      end

    end  endend

By simply modifying join_asset_file_contents to use our new function get_file_contents instead of File.read, we quickly get to the heart of the matter. CSS files get compress_css run on them, while Javascript files get JSMin.minimize run on them. Your :cache => true Javascript and CSS assets will now be gloriously combined and compressed!

Note that the above monkey patch requires jsminlib.rb, which you can download here. It is just a modified version of the original jsmin.rb, and you will want to put it into your Rails lib directory.

A good next step would be to further enhance get_file_contents to do Javascript obfuscation, which allows for the replacement of variable names and thus even further compression; it also tends to make Javascript code nearly incomprehensible and thus harder to steal, which may be desirable for some developers. I haven’t found any native Ruby ways to do this yet, but it seems to me that this would be a good place for a C extension (or similar), and that this should all be put into a tiny and lightweight plugin.

I’m always amazed at how easy it is to bend Rails (and Ruby) to one’s will, and in this case it’s really quite elegant and straightforward. I’d love to hear your ideas about how to take this idea forward, potentially even including it in Rails itself.

Download the files here:

• lib/jsminlib.rb
• config/initializers/javascript_minimization.rb

While I was working on some changes to Twittervision yesterday, I saw someone mention freerice.com, a site where you can go quiz yourself on vocabulary words and help feed the world. How? Each word you get right gives 10 grains of rice to, one hopes, someone who needs it.

The idea is that you will sit there for hours and look at the advertising from the do-gooder multinationals who sponsor it. Which I did for a while. I got up to level 44 or so and got to feeling pretty good about Toshiba and Macy’s.

It occurred to me though that my computer could also play this game, and has a much better memory for words than I do. In fact, once it learns something, it always chooses the right answer.

So I wrote a program to play the freerice.com vocabulary game. In parallel. 50 browsers at a time. Sharing what they learn with each other. Cumulatively.

It’s a multithreaded Ruby program using WWW::Mechanize and Hpricot. Nothing terribly fancy, but it does learn from each right and wrong answer, and after just a few minutes seems to hit a stride of about 75-80% accuracy. And a rate of about 200,000 grains of rice per hour (depending on the speed of your connection).

UPDATE: With some tuning, the script is now able to push out about 600,000 grains of rice per hour, which according to the statistic of 20,000 grains per person per day, is enough to feed over 720 people per day! If one thousand people run this script, it will (allegedly) generate enough to feed 720,000 people per day.

Before you go off on me, disclaimer: Yes, I realize this program subverts the intent of the freerice.com site. I’ve released this not to “game” freerice.com but simply to show a flaw in their design and have a little fun at the same time. If what they are after is human interaction, this design doesn’t mandate it. That’s all I’m saying.

Run it for a while and see how many people you can feed!

Prerequisites:

• Ruby (Linux, OS X, Other)
• Rubygems
• gem install mechanize –include-dependencies

Download the code

Yesterday, I received final confirmation that the Museum of Modern Art in New York has selected my mash-ups twittervision.com and flickrvision.com for its 2008 exhibition Design and the Elastic Mind.

I’m certainly very flattered to be included and have never considered myself to be an artist. I didn’t seek out MoMA on this. I am just very, very happy to have an opportunity to participate in a small way in the ongoing dialog about what technology means for humanity. Crap. Now I sound like an artist.

Incidentally, this means that twittervision.com and flickrvision.com are the first ever Ruby On Rails apps to be included in a major art exhibition. I already told DHH.

Anyway, at RailsConf Europe a few weeks ago, Dave Thomas’ keynote speech emphasized the role of software designers as artists. He said, “treat your projects as though they are artworks, and sign your name to them.” Or pretty close to it. I think this is incredibly valuable advice for software designers today.

We’re past the days of using machines as amplifiers of our physical efforts. It’s not enough to jam more features into code just so we can eliminate one more position on the assembly line. We’re at a point where the machines can help amplify our imaginations.

Today, creativity and imagination (what some folks are calling the right brain) are becoming the key drivers of software and design. With imagination, we can see around the corners of today’s most pressing challenges. While technical skill is certainly valuable, if it’s applied to the wrong problems, it’s wasted effort.

Creativity, imagination, and artistry help us identify the areas where we should put our efforts. They help us see things in new ways.

Everywhere I turn (perhaps partly because I am a Rubyist), I hear discussions of Domain Specific Languages, and of framing our problems in the right grammars.

This is hugely valuable because the creative part of our brain thinks in terms of semantics, grammars, and symbols. If we can’t get the words right, our imaginations can’t engage.

Everything stays stuck in the left side of our brains when we have to jump through hoops to please some particular language or development environment.

I hope you all will come out to see Design and the Elastic Mind when it opens at NYC MoMA, Feb 24 – May 12 2008. I’m not sure how we’re going to present the sites but we’re going to see if we can get some partners and sponsors involved to do something really beautiful.

And again, thanks to MoMA for the selection. And here’s to creativity, imagination, and artistry as the next big thing in software design!