{"id":75,"date":"2014-01-25T05:11:48","date_gmt":"2014-01-25T05:11:48","guid":{"rendered":"http:\/\/www.infosynergetics.com\/?p=75"},"modified":"2014-05-16T05:44:17","modified_gmt":"2014-05-16T05:44:17","slug":"sunspot-on-my-servers-makes-me-happy","status":"publish","type":"post","link":"https:\/\/www.infosynergetics.com\/?p=75","title":{"rendered":"Sunspot, on my Servers, Makes me Happy"},"content":{"rendered":"

After a week troubleshooting Solr and getting it running on Ubuntu for a test version of the Limspec<\/a> app, I think I am allowed the bad humor.<\/p>\n

I recently posted a summary<\/a> of how I deployed a Rails application on a Debian VPS using Nginx. \u00a0The steps are fairly similar to what you would do with Ubuntu. \u00a0For the Limspec project I\u2019m working on, we\u2019re using Apache – mostly because that is what I started with, but there is no practical reason to not use Nginx. \u00a0The instructions in that post should work well, but you would need to install the mod_rails module for Apache and create the appropriate sites-available files.<\/p>\n

The application that I was deploying is a fairly simple application for my Church\u2019s folk dance director to use for managing the dance program, and at this point doesn\u2019t utilize any search, let alone full text, so the instructions didn\u2019t include what is a fairly critical element to some of my other applications, and that is Solr for full text searching.<\/p>\n

Solr<\/a> is from the Apache Lucene project, and is a very powerful enterprise search platform. \u00a0I had implemented it for Limspec quite some time ago. \u00a0However, we had a VM meltdown a few months ago, and the replacement VM only seemed to have pieces and parts from the previous VM. \u00a0This was a huge problem getting Limspec deployed again (in addition to this, I no longer had root access, which is probably good as I was forced to set things up in a more secure fashion). \u00a0When all was said and done, however, I had forgotten to check Solr out. \u00a0It was working fine on my dev machine, so all search related tests passed with flying colors. \u00a0Important safety tip with TDD. \u00a0Even if you test exensively on your dev machine, you need to be very aware of those things that are deployed quite differently in production. \u00a0Solr is one of those. \u00a0Although it appears that you might be able to use the sunspot_solr gem in production, the developers who created it indicate they only intend it for use in development. \u00a0After hours spent trying to make it work, I tend to agree. \u00a0I could never quite get it running, so I finally gave up.<\/p>\n

So, not having taken notes on how I installed Solr the first time (well over a year ago), I set out to do it again. \u00a0Of course, there is a newer version of Solr, and a newer version of the sunspot_rails gem. \u00a0When I was rebuilding the actual Limspec server, I created an Ubuntu VM on my desktop to try everything on first. \u00a0So, I continued to use that VM to figure out Solr. \u00a0The following instructions are based on adding Solr to my local VM, which is running Precise Pangolin (Ubuntu 12.04 LTS).<\/p>\n

Jetty<\/h2>\n

Solr is a Java servlet, and so needs a servlet container of some sort. \u00a0Previously, I had used Tomcat. \u00a0However, Tomcat is fairly memory intensive and is really only necessary for more complex Solr installs (such as multiple instance). \u00a0Of course, if you are already using Tomcat for other purposes, it would probably make more sense to deploy Solr with Tomcat than to run yet another web server. \u00a0If not, Solr comes complete with its own copy of Jetty.<\/p>\n

From here on out, everything is fairly straightforward, but it took me a while to figure it all out.<\/p>\n

The first step is to download<\/a> the latest version of Solr, or at least the version you are interested in. \u00a0I opted for 4.6, which is the most recent version at the time of me writing this. Once you\u2019ve downloaded the tar file, you can untar it wherever you\u2019d like, as you\u2019ll be copying a subdirectory out to another location. \u00a0There are a number of locations you can use as your Solr home. \u00a0I recommend that you take a look ahead to the startup script from the Solr Wiki<\/a>\u00a0and choose one of the standard locations in the script. \u00a0Remember that I\u2019m looking to use Solr in support of another goal, so I want to minimize anything that makes my installation non-standard. \u00a0Taking this approach, as well, makes maintenance of the application and installation by other users, much easier. \u00a0I chose opt\/solr as my home, so I executed a mv solr-4.6.0\/example \/opt\/solr.<\/p>\n

The next thing to do is decide whether you are going to run single or multicore. \u00a0I tend to have staging instances on the same server as production, so I want multicore. \u00a0To deploy for multicore, within opt\/solr, delete the solr directory (i.e. rm -r \/opt\/solr\/solr), then mv \/opt\/solr\/multicore \/opt\/solr\/solr. \u00a0This then gives you a multicore deploy. \u00a0By default, you have two cores in place, core0 and core1. \u00a0You can certainly stick with those names, but I wanted names that would tell me what those cores are being used for. \u00a0If you want to change the names, first execute a mv core0 <newCoreName>, then update the solr.xml file within \/opt\/solr\/solr to indicate the new names and paths. \u00a0That is, change<\/p>\n

<cores adminPath=”\/admin\/cores” host=”${host:}” hostPort=”${jetty.port:8983}” hostContext=”${hostContext:solr}”>
\n<core name=”core0″ instanceDir=”core0″ \/>
\n<core name=”core1″ instanceDir=”core1″ \/>
\n<\/cores><\/p><\/blockquote>\n

to:<\/p>\n

<cores adminPath=”\/admin\/cores” host=”${host:}” hostPort=”${jetty.port:8983}” hostContext=”${hostContext:solr}”>
\n<core name=\u201cnewCore” instanceDir=\u201cnewCore” \/>
\n<core name=\u201cnewCore2″ instanceDir=\u201cnewCore2″ \/>
\n<\/cores><\/p><\/blockquote>\n

or whatever names you chose. \u00a0I tried to use names with spaces in them, and Jetty didn\u2019t like that. \u00a0I\u2019m not sure if it was because of the spaces, or the fact that the name wasn\u2019t the same as the instanceDir, so I just made them both the same and the problem went away.<\/p>\n

If you want to test your installation at this point, you can fire up jetty by running java -jar \/opt\/solr\/start.jar. \u00a0Then go to http:\/\/yourserver:8983\/solr, and you should see both of your cores in the coreadmin screen. \u00a0Note that you can change the port for Solr within the solr.xml file if you would like.<\/p>\n

Starting Automatically<\/h2>\n

The next thing you\u2019ll want is for Solr to launch on startup in the background. \u00a0This proves to be easy to do. \u00a0First, download the jetty.sh script linked to from the SolrJetty<\/a> page. \u00a0If you looked ahead and parked Solr in one of the standard locations, the script will work fine as is. \u00a0Place the script in the \/etc\/init.d directory and make it executable.<\/p>\n

Next, follow the instructions for creating the \/etc\/default\/jetty file for the various parameters jetty will need on launch, setting the Jetty home Java home, jetty user, etc. as appropriate. \u00a0If you opt to run it under a non-privileged user, such as solr (always a good idea), then follow the instructions on this page for creating the user and changing ownership of the solr directory. \u00a0Also set the user name correctly in the jetty configuration file (\/etc\/default\/jetty). \u00a0Finally set the run levels. \u00a0I just used the defaults (update-rc.d jetty.sh defaults). \u00a0I should note that every linux command you see on this page, I preface with sudo, as I\u2019m not operating as root. \u00a0More than likely this will be your situation, or should be.<\/p>\n

I always prefer to bounce my server after making a lot of these changes, to make sure that everything will start as it should. \u00a0So, I recommend doing that, then visit the solr admin page again to make sure everything is loaded.<\/p>\n

Configuring for Rails<\/h2>\n

As I stated before, this is for my rails application, so I need to do a few things to make that work. \u00a0I\u2019m assuming you\u2019ve followed something like this<\/a> to install sunspot_rails in your application. \u00a0If not, then do that. \u00a0Once completed, you will have a schema.xml file in your <rails_project>\/solr\/conf folder. \u00a0This needs to be copied into the conf folder for each core you are going to be using with your rails application (i.e., cp <rails_project>\/solr\/conf\/schema.xml \/opt\/solr\/solr\/core0\/conf\/schema.xml). \u00a0If you have an old schema.xml as I did, that predated Solr 4, you will be missing a key field definition, that needs to be added back. \u00a0Sunspot has been patched so that if you just installed it, you shouldn\u2019t have a problem. \u00a0If you get an error message about field _version_, then add this line in the fields definition section of schema.xml:<\/p>\n

<field name=”_version_” type=”string” indexed=”true” stored=”true” multiValued=”false” \/><\/p><\/blockquote>\n

Next, make sure your sunspot.yml file located in <rails_project>\/config is accurate with regard to port and path. \u00a0One thing that wasn\u2019t obvious to me, and sure is problematic is that the path is relative to the Solr directory. \u00a0That is, if your Solr directory is \/opt\/solr, and your core is \/opt\/solr\/solr\/core0, then the path in the yaml file should be \/solr\/core0. \u00a0The leading \/ is important as you will get an error otherwise.<\/p>\n

Finally, you will want to run a rake sunspot:reindex from within the your app directory. \u00a0If you get one of those great rake errors about having the wrong rake running, do a bundle exec rake sunspot:reindex, and all should be well. \u00a0I typically run a reindex on every deploy, just to make sure everything is good. \u00a0Sunspot will only index new and modified database rows, so if you want pre-existing rows to be searchable, then you need to reindex.<\/p>\n

My next step is to run all of this on our production server. \u00a0I\u2019ll post back an update on how that goes.<\/p>\n

UPDATE: \u00a0Ran this on the production server, and all worked as it should (provided you follow the directions, which I didn\u2019t at first, but that\u2019s another story).<\/p>\n","protected":false},"excerpt":{"rendered":"

After a week troubleshooting Solr and getting it running on Ubuntu for a test version of the Limspec app, I think I am allowed the bad humor. I recently posted a summary of how I deployed a Rails application on a Debian VPS using Nginx. \u00a0The steps are fairly similar to what you would do […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[],"_links":{"self":[{"href":"https:\/\/www.infosynergetics.com\/index.php?rest_route=\/wp\/v2\/posts\/75"}],"collection":[{"href":"https:\/\/www.infosynergetics.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.infosynergetics.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.infosynergetics.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.infosynergetics.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=75"}],"version-history":[{"count":2,"href":"https:\/\/www.infosynergetics.com\/index.php?rest_route=\/wp\/v2\/posts\/75\/revisions"}],"predecessor-version":[{"id":158,"href":"https:\/\/www.infosynergetics.com\/index.php?rest_route=\/wp\/v2\/posts\/75\/revisions\/158"}],"wp:attachment":[{"href":"https:\/\/www.infosynergetics.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=75"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.infosynergetics.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=75"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.infosynergetics.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=75"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}