After that, it literally took 10 minutes to create a view for uploading a file, connect it to the controller, and create another view for displaying the uploaded file.

(Specifically–uploading the location CSV to be parsed and added to the database)

Now to parse!

I’m really beginning to fall in love with Ruby on Rails. It’s been kind of difficult wrapping my head around some of the concepts, but once I do… damn it’s fast.

Once I get the CSV upload mechanism complete, it’s time to create another action for searching and returning a set of locations (in JSON!)

Pretty easy once I figured out the limits of my host.

So far in the Ruby on Rails/PHP-Yii contest, RoR is winning handedly. Parsing text is almost absurdly easy with Ruby.

So then I started playing around with Ruby on Rails again. Now with a bit better understanding of how it all works, it… well frankly, it’s fun. Ruby is a joy to program in, and Rails makes certain things really slick and easy once you figure out what the conventions are.

So right now I’m coding my first task (importing the zip code database) in parallel in RoR and PHP/Yii. We’ll see which one wins.

I’m still trying to decide which language to use for the service portion of the backend. Simply from the writing-a-daemon perspective, C would be the logical choice. However, a good secure Telnet client library and a good MySQL implementation are already important (yes, I know I could write my own, but I’m a firm believer in not repeating the work somebody else has done unless I can improve on it). Everything else can be tacked on as needed, but at this point the service portion of the backend consists pretty much of something to read the telnet feed, something to get the info into the database, and something to send the push notifications. I am thinking about using GNUStep to kind of keep things on the same wavelength .

After making a design diagram and really looking into what I need the backend to do, it became very clear to me that there are three main functions of the backend: collect data, push alerts, and respond to on-demand requests for data.

Items one and two are linked–I need to send a notification out as soon as the data comes in. However, item three can ostensibly exist on it’s own, separate from the other two. As I sat and thought about it, this made more and more sense. When I was first designing the system, WeatherMan had its own protocol for communicating with the server. As I was working, I ended up ditching that for a demand/response system over HTTP, which I kept integrated into the monolithic server.

It’s become apparent to me that this is a waste of resources. I will probably keep the WeatherWire parsing engine/notification system in Java since it’s already written there, but I will be offloading the demand/response system to either PHP or Ruby on Rails, to take advantage of the web server systems that I’m already running anyway.

Yay progress!

Any Ruby resources you all can recommend?

• There was a nasty infinite loop somewhere in the code that I was unable to nail down the root cause of; some condition in the WeatherWire parser was causing the issue
• I need to write a truly secure communications channel between phone and server; location information is stored and transmitted, and I want to make sure that is secure. Last year I was using a hashed and salted device token to authenticate with the server. However, the communications channel itself was unencrypted, and this required that the device send a token received from the APNS server, which is something I’d like to avoid doing.
• The backend was never really set up as a daemon; start/stop was manual and there was no external control.

Because of the significant rewrites required, I’m probably going to end up breaking the whole thing down, reusing the code that works, but rethinking the architecture. This got me thinking–is Java really the best language to use for this?

Here is what the backend must do:

• Keep a consistent connection via telnet to the WeatherWire server to pull down weather alerts
• Parse the data coming into the telnet connection to decide what is an alert and what isn’t, and to break the alert down into an object that can be stored in the database
• Keep records of all devices using the software and the locations they have requested to receive alerts for
• Each time an alert comes in, find the location of that alert, match it to the device database, and send an APNS notification to each device that wants to receive alerts for that location
• Connect to the NWS FTP servers to download data on-demand
• Act as a cache for on-demand information (forecasts, radar images) to reduce loads on the NWS servers
• Insert and retrieve all of this data from a MySQL database
• Expose an HTTP server over an SSL/TLS connection that will take POST requests for data and return JSON datasets (there shouldn’t be any actual binary data required, as even the radar imagery will be in a KML format that is parsed on the device)

So basically, whatever language is used needs to have good networking libraries and good text/string processing capabilities, and needs to be fast enough to process the WeatherWire data in realtime (there can be quite a bit of it). Also, the backend must be able to run on a Linux server, which immediately counts out Objective-C/Cocoa and .NET for the backend. Here is the list of languages that I’m considering, and what I feel the pros and cons are. Any comments would be appreciated.

• Java. Via the String class, Java has strong enough text-processing capability to meet my needs. Also, it has a large amount of open-source libraries available; I made extensive use of the Apache Commons libraries in the original version to implement telnet and FTP connections, and used the embeddable Jetty webserver to have a lightweight HTTP server to handle the JSON requests. Plus, while the JVM isn’t as fast as native code, it is generally going to be faster than an interpreter.
• C/C++. C is a little lower-level than I’d like to use for this application; being able to take advantage of class libraries that will handle the finer points of network connections and string processing is something I would miss dearly. This option would definitely win on speed, however the learning curve and heightened security risks make me wary.
• PHP. PHP would be the clear winner (at least in my experience) for the text processing and HTTP responses. However, the interpreted nature could cause slowdowns, and while I’ve seen a lot more non-Web platform PHP use recently, I’m not sure if the tools are all there (anybody ever seen a Telnet library written in PHP?). Plus, can it be easily daemonized?
• Python. I have no experience with Python, but I do know it has a fairly extensive standard library and a lot of developer support. The interpreted nature and potential difficulty daemonizing are a couple of potential pitfalls.
• Ruby. Another language with fairly extensive developer support that I do not have any experience with. Again with the interpreted nature and daemonization, though based on the Rails framework I could see some ease with the server portions of this.
• Go. Google’s Go intrigues me as a language. It seems to me that it would offer a lot of the benefits of C/C++ without a lot of the pitfalls (namely direct memory access). The newness of it is a turn-off though–specifically, there is no native connector for MySQL.

If you have any thoughts or insights I’d love to hear them. I’m up for learning something new if it will meet my needs in a better way than Java currently is.

1. Approval by the Ethics and Compliance office of my full-time employer took nearly six months;
2. The mapping API built into iPhone OS 3.0 does not support overlays in any way, shape, or form, which leads into
3. I have a full-time job. Between the requirements (read number 1) and the lack of time to code my own map solution for radar (read number 2) it seemed pretty silly to continue dumping time into a project that at that point seemed like it may never see the light of day.

Number one on that list is being worked out. Number two… I can’t talk about. And number 3, while it still exists, is about to become less of a factor as I’ll be shifting into a 4-10s schedule, giving me an extra free day each week, as well as the work being somewhat less mentally exhausting (anybody that’s ever done customer-facing technical support knows exactly what I’m talking about).

As I did previously, I plan keeping tabs of my development here for anybody that’s interested to follow. I will not be starting from the ground up; I have a lot of good code in what I’ve already worked on, which I plan on cherry-picking and utilizing as I rewrite the parts that I feel did not work as well as they could have.

In case you couldn’t tell, the release target OS for the app is iPhone OS 4. Unfortunately, the developer NDA prevents me from saying anything that hasn’t already been publicly stated, but I can give you a little teaser from what -HAS- been shown:

(Image lovingly ripped from Engadget)

I’m looking forward to making some progress on this application this summer, and I hope you all enjoy the ride as I think out loud on this blog.