Chronicling My Ruby on Rails Journey

Pondering Ruby on Rails Scalability

Posted by: Bob Ngu on: January 26, 2008

It has been a while since I blogged. My site, JiggyMe has been chugging along and has picked up as much traffic as 50,000 hits a month, 3000+ hits a day. I am hosting my site on Slicehost 256MB, 10GB VPS and it has held up rather well so far. However, I keep hearing how Ruby on Rails can’t scale, courtesy of the constant downtime at Twitter.

I know that Rails 2.0 has recently been released but unfortunately I haven’t been paying much attention to it. I wouldn’t mind offering my site up for scalability experiments. Rails is fairly young and I believe that as more sites encounter scalability issues that Rails will be extended to address those issues. Scalability is a complex domain and there is no one correct solution. If the limitation of Rails not being able to connect to multiple databases is the scalability problem, then I believe it is a solvable problem.

How about it, Rails developers, shall we prove the naysayers wrong? Afterall, scripting languages were never designed to scale and yet they (Perl, Python, PHP) are scaling now, it goes to prove that it is just a matter of time to work out the scalability issues.

Tags:

Adding SMTP mail server support to your deployment

Posted by: Bob Ngu on: June 22, 2007

My RoR efforts have culminated in JiggyMe, check it out!

As one of my final steps in deploying my rails app, I have to setup mail for my domain. I ended up using foldera.com free MX servers for sending and receiving mail, but foldera does not allow mail relay with their servers. Foldera has closed shop and cut off support for mail server. After a bit of research, I found Google Apps as the replacement. This is highly recommended, so check it out if you don’t want to pay / maintain your own mail servers.

So what to do because I need to send email, not spam mind you, from my rails apps? A simple and quick solution is to install postfix, in my case, “yum install postfix” on FC6

The only change to /etc/postfix/main.cf (Postfix config file on FC6) you need to make afterwards is
mynetworks = your_ip/32, 127.0.0.0/8

or:

mynetworks_style = host

That configures postfix to trust only localhost.

The corresponding change to your rails app is to config/environments/production.rb or development.rb etc
ActionMailer::Base.delivery_method = :smtp
ActionMailer::Base.server_settings = {
:address => “localhost”,
:port => 25,
:domain => “yourdomain.com”
}

I am only using this setup for outbound SMTP mail only, nothing else. My server is already secured with a firewall that blocks all incoming traffic except for selected open port (25 not being one of them), so this setup should be fairly secure from being exploited by spammers.

Successful deployment using Capistrano from Windows to FC6 on slicehost

Posted by: Bob Ngu on: June 13, 2007

My RoR efforts have culminated in JiggyMe, check it out!

After having spent 3 weeks and hitting practically every bug or issue possible, I have finally successfully deployed my Rails app from my Windows XP development machine to FC6 slice using Capistrano. I am using the now seemingly standard configuration of Apache 2.2 with mod_proxy_balance talking to a Mongrel cluster. Other FC6 bits include Subversion, SSH with PKI, svn+ssh, and MySQL. I have learned a lot from this experience but it has been especially painful for me since I am a newbie at Linux, ssh, Subversion, RadRails, Ruby, Rails, MySQL, Capistrano, Apache, Mongrel, hell you name it, I am new to it.

Regardless of what others have said, getting deployment to work the first time is no small feat unless you are using Ubuntu and going with deprec gem. I decided to go with FC6 and had no idea the trouble I was about to get myself into. Making it worse is that I am deploying from Windows which is by itself not well-documented because most deployment are from *nix to *nix systems.

I would be glad to share my experience here with anyone going through the same pain as me, just post comments and I will do my best to answer them.

P/S: I am in the process of releasing my website and could use some enthusiastic Ruby on Rails developers (perhaps even co-founders?), especially those in Bay Area, CA, that are interested in further developing the website. Drop me a comment here or send email to support at jiggyme dot com and I will get in touch with you, maybe we can work something out.

Slicehost FC6, Subversion, svn+ssh, RadRails with PKI

Posted by: Bob Ngu on: May 31, 2007

My RoR efforts have culminated in JiggyMe, check it out!

UPDATE 6/5/2007: Finally got SVNKit to work with PKI. The malformed network error is due to the fact that I echoed “using svnwrap…” on the server side while running svnwrap wrapper, ARGH!. It works now for both Eclipse and Radrails standalone. Enjoy!

UPDATED 6/1/2007:
As it turned out, my problem below with SVNKit is because I was using Putty’s private key format (PPK). SVNKit only supports openssh format. To convert Putty key (PPK file) intoOpenSSH one please use puttygen.exe program. Load your private key into it and then use “Conversion | Export OpenSSH Key” from the main menu. With that and SVNKit, I moved past my original problem but now I am onto a new error, “SVN: malformed network data”, DODH! Anyway, my suggestion below with using JavaHL still works. But I am in the process of trying out Eclipse with Radrails and Subclipse plugins instead of continuing to use standalone Radrails. Radrails has been abandoned and has since been repackaged and picked up by Aptana and there are numerous issues with Radrails 0.7.2 with the latest 1.4.x Subversion, see here for details. Apparently Eclipse with Radrails and Subclipse 1.2.0 suffers the same malformed network data error problem, oh well.

Original post:
After several days of research, experimentation, pain, and suffering, I managed to finally get RadRails 0.7.2 with Subclipse plugin 1.0.5 working with Subversion 1.4.3 on FC6 slice on slicehost. And what a pain it was to get it working, the major pain being the part getting PKI working correctly with RadRails, Subclipse, and Subversion. A couple of points worth noting that caused me a great deal of pain

  1. in Radrails, use JavaHL (JNI) SVN interface and not the SVNKit (Pure Java), this can be changed in Radrails…Window…Preferences…Team…SVN dialog. As it turned out SVNKit works fine with username/password authentication but not with PKI. When I tried to use SVNKit with PKI, it simply re-displays the authentication dialog after I entered the PKI details. With JavaHL, you have to also install Putty and use Plink as the external program launched by JavaHL, see article here for config. This isn’t pretty as it launches a Windows command box everytime it talks to Subversion but hey would you rather have something that works and ugly or something that doesn’t work? It is useful to also have Pageant running so you don’t have to enter the passphrase everytime.
  2. In Subversion, it is best to create a wrapper script, svnwrap, to use with all SVN executables. This is to set the correct umask since it isn’t a guarantee that all processes write to the database files with friendly umask, as demonstrated here and here.

This was part of my effort migrating Subversion from my laptop to slicehost, here’s an article on how to do that. Enjoy!

Ever wonder how to write to Rails log files?

Posted by: Bob Ngu on: April 12, 2007

My RoR efforts have culminated in JiggyMe, check it out!

You can do it like this using RAILS_DEFAULT_LOGGER.error, e.g.,

RAILS_DEFAULT_LOGGER.error("\n test \n")

This writes to one of 3 logs: development, test, production depending on what mode you are in when you executed the code. Enjoy!

Updated 10.30.2009
From one of the commenter

Since rails 2.1, there’s a more convenient way to log:

logger.debug ‘message’

You can use the following methods on logger: debug, info, warn, error, and fatal.

Strings and longer strings

Posted by: Bob Ngu on: April 10, 2007

My RoR efforts have culminated in JiggyMe, check it out!

So if you have a long string, you can use “/” for line continuation, e.g.,

str = "this is line 1" /
        "this is line 2"


Or an alternate way of specifying double-quoted string literals, use %{…}, e.g.,
str = %{this is line 1
            this is line 2}

Subversion 1.4.2 server and Using it from Rails and RadRails

Posted by: Bob Ngu on: April 10, 2007

My RoR efforts have culminated in JiggyMe, check it out!

Ok, so here I am, a Linux newbie, a Subversion newbie and more or less a RoR newbie as well trying to figure out how to setup and configure Subversion for use by Rails project from RadRails using Subclipse. This is in preparation for migrating my Rails apps to Slicehost in the future, I had no idea what was in store for me.

Anyway, I am writing down all the newbie steps I remember and if it helps someone else, great. Note that this is from the perspective of a Linux and Subversion newbie…

First off, my Win XP environment is setup with the following

  • RadRails 0.7.2 with Subclipse plugin for access to Subversion
  • Ruby on Rails installed with MySQL and using Webrick for development

My Linux environment is as such

  • Fedora Core 6 on VMWare Server 1.0.2
  • Subversion 1.4.2 on FC6.

Some basic things about Subversion (SVN):

  • SVN runs on port 3690, so make sure your firewall is configured to allow access
  • to show svn server version “svnserve –version”, for svn client, “svn –version”
  • svn can be accessed remotely using Apache, svnserve, sshd over svnserve (Chapter 6 of Subversion manual). I am using the sshd over svnserve method
  • to run SVN server, run “svnserve -d”, you can restrict access with path like “svnserve -d -r /home/svn/repository” and you can access your project using “svn:///project1″. You can also run SVN using inetd, for those instructions, read the manual
  • to list repository contents, “svn list file:///home/svn/repository/trunk”
  • there isn’t a way to gracefully shutdown svnserve, just do “ps -e | grep svn”, find the process id, and kill it

Now the install steps on FC6:

  1. Install subversion server: “yum install subversion”
  2. Create 2 SVN users, “adduser svn”, and “adduser user1″
  3. Add passwords both SVN users, “passwd svn”, and “passwd user1″
  4. Add user1 to svn group, and set rwx permissions for svn group to repository “chmod -R g+rwx *” recursive starting with svn directory.
  5. Create a new repository using svn account “svnadmin create /home/svn/repository”
  6. Create a basic directory structure for the repository. I use the recommended three top level directories: branches, tags and trunk. I work out of trunk and create branches when I have stable builds. Create this structure in a temporary directory (created below) and then import it into the new repository.

    mkdir ~/temp
    mkdir ~/temp/branches
    mkdir ~/temp/tags
    mkdir ~/temp/trunk

    Do the initial import of this structure. Note that the location of the repository must be explicitly stated—you can’t use ~/. Of course, all ~/ means is /users/your_user_name. Once you’re done, you can trash that temporary directory you created (be careful with that rm command!).

    svn import ~/temp file:///home/svn/repository/ -m “initial import”
    rm -r ~/temp

  7. NOTE: I have decided to use repository structure with branches, tags, and trunk at the root and project name underneath. There are other ways to structure your repository but this is a fairly common structure.

Now, to get RadRails to talk to SVN (all steps on Win XP machine):

  1. to get RadRails to talk to ssh, under Windows –> Preferences –> Team –> SVN –> SVN interface: make sure you have selected JavaSVN.
  2. from within RadRails, add a new repository, follow the instructions here
  3. from RadRails, right click on your project…Team…share project, select SVN, use existing repository from above, use project name as folder name. When I did this initially, RadRails gave an error while trying to import the project “malformed network error”. Follow the instructions here to fix the problem.
  4. You should ignore log files and tmp files from svn, see here and here for more details

Phew, now you should be all set, good luck!

Walking the REXML tree

Posted by: Bob Ngu on: April 1, 2007

My RoR efforts have culminated in JiggyMe, check it out!

This took me a while to find out but if you ever want to walk the tree from any entry point, use the each_recursive method. An example code snippet to do so ….

#Recurse end to end, printing the tags
		@doc.elements.each("definitions/src") do |element|
			print "[", element.name.to_s, "]"
			element.each_recursive do |childElement|
				print "[", childElement.name.to_s, "]"
			end
		end

The REXML documentation does not make it easy to figure this out. The reason the method is available is because Element inherits from Parent which inherits from Child. In child.rb, there is a “require\node” statement. node.rb is where the each_recursive method is defined. And yes in case you haven’t noticed, the way to get an a tag element’s name is via the name() method.

I got the example above from the following article . There are other examples duplicated here for your convenience.

If you just want the next level of children, but no deeper, I’m not
sure what you call. I did this when I played with REXML, and the
obvious each_child doesn’t give you an REXML::Element. It gives a
REXML::Text element at the first iteration, then the next
REXML::Element, then another REXML::Text object, etc. Not quite what
you want. But adding this to your code will work.

module REXML
	# Visit all children of  this node, but don't recurse to their children
	def each_child_element(&block)
		self.elements.each {|node|
			block.call(node)
		}
	end
end


It probably exists in some form in the REXML module, but I can’t
find it, so I recreated it (by a little hacking of the modules
each_recurse).

You can then

		#printing the tags of the immediate children.
		@doc = Document.new(File.new(format_file))
		@doc.elements.each("definitions") do |element|
			element.each_child_element do |childElement|
				print "[", childElement.name.to_s, "]"
			end
		end


or recursively walk the tree by calling each_child_element for each
returned childElement (as with the first example)
	def recurse(the_element)
			the_element.each_child_element do |childElement|
				print "[", childElement.name.to_s, "]"
				recurse(childElement)
			end
	end

	@doc.elements.each("definitions/src") do |element|
		recurse(element)
	end

Global layout

Posted by: Bob Ngu on: March 26, 2007

My RoR efforts have culminated in JiggyMe, check it out!

Just in case you don’t already know, the way to use a global layout is to place a file called “application.rhtml” in the layout directory. If you have a matching controller layout file, it will override application.rhtml.

Where do I put static files like images, stylesheets and javascripts?

Posted by: Bob Ngu on: March 24, 2007

My RoR efforts have culminated in JiggyMe, check it out!

Static files should go in the public folder. It is best to put images in public/images, stylsheets in public/stylesheets and javascripts in public/javascripts. This will allow you to use the built-in Helpers more effectively. In your rhtml, the url’s would be relative to public. Thus it’s /images/image.png not /public/images/images.png.

This really threw me for a loop as I tried to figure out how to refer to the images from CSS, lesson learned.

Older Entries
Newer Entries

Blog Stats

  • 36,445 hits
Subscribe in a reader

Subscribe to Chronicling My Ruby on Rails Journey by Email

Recent Readers

View My Profile View My Profile View My Profile View My Profile View My Profile

Blog at WordPress.com.

Albeo theme by Design Disease