Hank Beaver

I’m am still captivated by the perfect, simple CMS solution in Ruby/Rails (I’ll take another framework too). There are some fantastic projects out there, browsercms.org for example. But posts like THIS , really drive home the point. If Wordpress is so well suited for a simple site and/or blog, why not just use it? Igvita demonstrates easy Rails integration with Apache rewrite rules.

There are others out there stating the same thing: while Rails can be used to create a CMS, a developer’s time is better spent solving business rather than reinventing the wheel in Ruby to keep the whole project/code under one umbrella. And at my current gig, we are starting to retool for this paradigm. Rails and Sinatra apps are purely business focuses and super light weight. These service applications can them be glued together via ESI, rewrites, or Wordpress for consumption. In effect, your complex web app is assembled at request time from a set of simple web services. Here is some mock code to demonstrate:

wordpress_template.php
//some more wonder wordpress stuff
<?= include_rails_service('http://service.app/user/profile',ESI_INCLUDE) ?>
//some more wonder wordpress stuff
<?= include_rails_service('http://service.app2/events/weeks/2',SERVICE) ?>
//end of wordpress.php
...
//function included elsewhere
function include_rails_service(service_url,type){
//could return any of following for the resource url
 switch (type) {
  case ESI_INCLUDE:
   return "<esi include ='" + service_url + "'>";
   break;
  case SERVICE:
   include service_url;
   break;
  case AJAX:
//javascript include with service js, doing doc.write
}
}

The point is Wordpress doesn’t know about Rails and Rails doesn’t know about Wordpress. Keep them seperate, keep them simple and make life better.

When using a BigIP to load balance a Sinatra app via Thin you will need to provide more than just the GET request string in F5’s “send string” field OR you might get “Request Invalid!” error message. Heads up.

Using this will cause issue:

GET /ops/heartbeat

This works:

GET /ops/heartbeat HTTP/1.1\r\nHost: \r\nConnection: Close\r\n\r\n

Simple but effective regex to monitor when there are issues. In my example below, this will show all response times > 1 second:

 tail -f production.log | egrep "Completed in [0-9]{3,10}"

Introduction

At my current gig with Primedia, we have some pretty high volume Rails sites. Since having highly available sites are essential for our core business, we have to be very attentive to production issues. Below is a list that I’ve compiled recently in my head and by working with some very bright folks. This doc is to help Ops/Dev remember some basics and tricks when debugging issues. So remember to try this stuff.

Non-Technical (READ FIRST)

1. If the issue is not a major impact on production, make sure all your troubleshooting is passive! You might introduce an outage if you try serious troubleshooting.
2. Confirm the issue with Ops/Infrastructure in your presence, this gets buy-in from with the folks who have official root access.
3. As a developer, you should not work alone on production systems except to gather information (if you get temporary access). All changes (in seeking issue mitigation) should be paired with Ops/Infra. This will avoid finger-pointing and headaches later.

Technical

Elimination + log files is my troubleshooting method of choice.

1. Skip the load balancer if possible and hit direct IP/Port of a single app server to eliminate load balancer in mix.
2. Enable DEBUG log level and restart app server(s). Preferably, take an app server out of load balancer pool and work on in isolation. Regardless, DEBUG can slow response signficantly so be aware. However, there are situations where DEBUG is the only way to get an indication from Rails what is going on.
3. Tail log file of single app IP/Port to squelch noise from other servers and hit just that app server.
4. Make sure that the PIDs are not hung,old versions. Look at time/date of PIDS. If the deploy is screwed up, you can be deploying new code and yet the processes running are code from a month ago.
5. To be thorough, look at the kernel logs too. In Linux, /var/log/messages etc. You could have something else that is going on.
6. The logs should give status regarding database(s), but especially look for the words “timeout” if you have any custom network libraries that fetch data (we do) or “ActiveRecord”.  Perhaps even, grep for it.
7. Use a non-browser client, like Curl or wget to eliminate possible browser issues. Case in point, we had an issue related to ActiveRecord sessions that were migrated to another datacenter. When accessed via Curl there was no issue (Curl does not save cookies by default).

Other

1. If you cannot duplicate issue in development, remember to change your configs to look like production mode and run your development workstation in production mode. Rails behaves different in its modes.

Ironically, the two most interesting items for me personally were not directory related to Merb at all. I was quite impressed with the CSS capabilities of SASS and CouchDB gave me hope that being stuck in the rigidity of columns in SQL is not always my path. Here’s my my Merbday summary,  iteration retrospective format:

STOP:

• Assuming Merb is “fragmentation” (so said someone smarter than me ) of the Rails community and a real effort to improve on Ruby as the defacto web programming language.

START:

• Using ruby-prof in Rails projects and Kcachegrind to profile apps. (thanks for Yehuda Katz at www.engineyard.com )
• Playing with Merb on a real project and focus on understand slices and interaction with other slices.
• Investigate writing an Admin or basic CMS slice with someone else.

CONTINUE:

• Loving Rails and not abandon it as yet.

Motivation:
As an avid Ruby/Rails programmer and longtime CMS coder and user the following is yet another effort to keep a list of current Ruby/Rails CMS’s. That is the goal of this post. I will use comments from others to update the list and hopefully keep and up-to-date list for all (including myself) to use. Heck, I would love to have others help review and keep list up to date.

External Resources:

http://www.widgetfinger.com/
http://www.ajaxlines.com/ajax/stuff/article/top_ruby_cms.php

http://webscripts.softpedia.com/downloadTag/ruby+cms

We had an issue the last couple of weeks on a new Rails project we started here at my gig. All Rails exceptions were quietly logged without showing any stack trace to browser. This was very annoying. Secondly, the RAILS_ENV was reported as “test”. mmmm, very suspicious. After some investigation I found the culprit on rails.rb line #1 of rails-rspec gem installed in our vendor dir:

Take a look at this to get a little context:

http://ryandaigle.com/articles/2008/4/1/what-s-new-in-edge-rails-gem-dependencies

How can rspec affect your actual Rails environment, script/server, script/console, script/about? I’ll show you. enter our environment.rb:

Aha! So effectively, by including our gems here too (and gaining the added benefits noted above at ryandaigle.com )rather than simply in our spec_helper, we replaced our RAILS_ENV and invoked the Rails equivalent of an exception black hole.

Passenger (a.k.a mod_rails) is getting some buzz (google it). Looking at the below config and the fact that deployments are automatic from Web Server with a simple file touch.

In http.conf:

LoadModule passenger_module /usr/lib64/ruby/gems/1.8/gems/passenger-2.0.3/ext/apache2/mod_passenger.so
PassengerRoot /usr/lib64/ruby/gems/1.8/gems/passenger-2.0.3
PassengerRuby /usr/bin/ruby

<VirtualHost *:80>
ServerName vwebadm11.atlis1
#DocumentRoot /var/www/apps/html
DocumentRoot /var/www/apps/provisioning_webservices/current/public
ErrorLog logs/prodops_errors_log
CustomLog logs/prodops_log combined
#RailsBaseURI /prov_websvc         # Passenger Rails link
RailsEnv test
</VirtualHost>

<Directory "/var/www/apps/provisioning_webservices/current/public">
Options FollowSymLinks
AllowOverride None
Order allow,deny
Allow from all
</Directory>

When instantiating a model class this error occurs:

instance = MyModel.new

NoMethodError: You have a nil object when you didn't expect it!

The error occurred while evaluating nil.has_key?

This is rectified by placing a super within model class.

def initialize

super

logger.debug("initialize VirtualMta: #{self.class}")

end

Goals:

• To auto start with init.d, upon westhost restart, etc.
• Use Apache to front end, via mod_proxy.

You’ll have to follow other sites on installing gems, etc. This Site has some very useful info, namely you need to symlink mongrel_cluster_ctl and ruby to /usr/bin/ (you’ll have to do a find / -name mongrel_cluster_ctl). Mine was here: /usr/local/ruby/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/bin/mongrel_cluster_ctl

So I linked into /etc/init.d/ like so:

ln -s /usr/local/ruby/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/bin/mongrel_cluster_ctl /etc/init.d/

chkconfig is not installed on westhost, so you have to do the init.d link/scrips the old fashioned way:

ln -s /etc/init.d/mongrel_cluster_ctl /etc/rc.d/rc3.d/S81mongrel_cluster

ln -s /etc/init.d/mongrel_cluster_ctl /etc/rc.d/rc3.d/K19mongrel_cluster

K = Kill, the number means the order

S= Start, “”