Skip to main content

Smarter project configuration with Vagrant

There are a few tools in programming that have made my life much easier by keeping the “annoying” parts of coding out of the way. I love writing code, but I don’t love dealing with dependencies and their interactions. In a perfect world,  making sure that different projects don’t get in each other’s way should not require great effort. Vagrant is a tool that helps us get a step closer to such a world. There are other tools of course, but this tutorial is about Vagrant, so we won’t digress.

What is Vagrant and why do you want it?

Vagrant is a tool made by Hashicorp and it allows one to abstract virtualization. Virtualization, in this case, is the idea that you can run a secondary virtual operating system (a guest machine) on top of your own (a  host machine). You can use a virtualization software, like Oracle’s Virtualbox, on your windows machine to run a copy of Linux like you would any other program. Yes, I am starting my blog ripping off the plot of Inception, this is indeed the story of OSes inside OSes, and then some. Just to give you a break from words in favor of pictures, here is what VirtualBox looks like on a windows machine:

 

Vagrant takes things a step further. It can be configured to use different virtualization software and it lets you set up configurations for these in a script, called a Vagrantfile. You get the benefits of using a Virtualization tool, but you also get a bit more leeway in terms of how and where you virtualize. To start, you could use Vagrant with Virtualbox right out of the box, but you could also use say… Amazon cloud or fancy tools like Docker. Any of these and more can be used as a Vagrant “provider” and you would additionally get some useful Vagrant enhancements on top of them. How would this be useful, you ask?

  • You can test a virtual machine out locally, and flesh out a server on the cloud with very few changes and very quickly.
  • You can build and tear down machines with the specified configuration rapidly without having to go through a checklist for the setup each time.
  • You can share configurations across the web with a single script, and you can also download pre-built machines people have prepared and shared as part of their work.

Speed is a common theme here because that’s part of what Vagrant gives you.  Vagrant allows me to match remote machine specs, spin up quick test machines, deploy successful machines to the cloud, and share my working environments via tiny script files. If I need to test a C++ program on a specific machine/compiler combo, it takes about 5 minutes to set that up. In fact, I can save the recipe and use it later down the road by simply saving the Vagrantfile for it. I maintain my local/development environments for this website through Vagrant (along with a couple other tools). I can test changes locally, and push them to a near-identical machine with very little work. If I am working with a team, changes to my script over source control updates their machine tools the next time they pull from the repository.

Before the gain, comes the pain that is the install. Thankfully, this is also pretty quick for Vagrant. Things change with time, and the install is where this tends to get me, so please feel free to ask questions, or share insights through this process.

How do you get Vagrant up and running on your machine?

This really depends on what operating system you’re running. I’ll need you to have OpenSSH and Rsync to get things started. These should already be part of your machine if you run OSX or Linux but if you use windows, you will need to prepare your machine a little.

we might not get to using ‘rsync’ in this tutorial, but I am having you install it because it will likely become useful in future projects, so do it for my sake for now.

For Windows:

If you run windows, you will either want to run BASH on windows 10 or download Cygwin. I am going to go through the second method because I have that working and it’s the bigger pain in the butt. Cygwin lets you emulate Linux’s bash in windows through its own console and gives access to the set of tools that you would see in a Linux or MacOS terminal.

Here is what we need to do:

Installing with Chocolatey

Chocolatey is a package manager for windows. You can get it at the chocolatey website. Just find the latest directions for installation and follow the steps to install it. The details change from time to time, but the gist of it is:

Open Powershell in admin mode, make sure your Get-Execution policy is not set to restricted(you can learn more about their security policy from their own site and find more information here), and then run the command below:

iwr https://chocolatey.org/install.ps1 -UseBasicParsing | iex

Once this install is successful, you now have access to the cinst or choco install commands, which let you call on the chocolatey installer. Now we need to install Vagrant and Virtualbox:

choco install vagrant virtualbox

We now have Vagrant and Virtualbox. Next grab Cygwin like so:

choco install cyg-get

cyg-get is a chocolatey package that lets you install add-ons through PowerShell, which is what we’re here for, so type in the following commands in sequence:

cyg-get openssh

cyg-get rsync

Now we should be ready to go. If you type in ‘Cygwin’ through the windows search, you should be able to open it. I always use admin mode. You can also find the folder for it on your primary drive (usually C: under the folder named tools. This will be useful so take note. I’ll do most of my work in C:\tools\cygwin\home\myusername. You can work elsewhere of course, but just be consistent.

 

I should mention here that a while back I ran into an issue with my install. IF you run into issues starting up cygwin at this point, try this:

  1. Download the setup installer on the cygwin website: https://cygwin.com/setup-x86_64.exe
  2. Rename the installer to cygwinsetup.exe
  3. Move the installer to C:\tools\cygwin (or wherever you install cygwin)

Let’s test out Cygwin. Open it up and test each of the programs we installed by typing

ssh

and then

rsync

You should see messages corresponding to each tool and how to use it. You should NOT see something like “command not found”. That would mean something did not install correctly. To summarize, we’ve done the following:

  1. Install a Package Manager called Chocolatey (because package managers keep things organized, but you can skip this if you feel daring)
  2. Install Cygwin
  3. Make sure we have Rsync and OpenSSH installed.

For Mac OSX machines:

You can similarly use a package manager on a Mac (like brew) or you can install directly. We’ll use brew.

Installing with Brew:

Open up Terminal, and type in:

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Then we need to tap the Cask repository like so:

brew tap caskroom/cask

and then install the cask command:

brew install brew-cask

Finally, we can installVagrant and Virtualbox

brew cask install vagrant virtualbox

You’re all set. Moving on

For Linux machines:

The Vagrant and Virtualbox websites will walk you through the choices of installing on Linux. I prefer the sudo apt-get methods, but I only tested on Windows/Mac so you’re on your own here. Generally, if you’re a Linux user, you’ll want to do this your own way depending on your preferred packages etc. Just be careful, I’ve heard some packages have really old versions of Vagrant, and you want to use a somewhat recent version. If you can’t figure it out, let me know. I’ll get off my lazy behind and test something out for ya.

“Hello world”

In this section, you will learn the basics of Vagrant and be given the minimal tools to make use of it.

We should now have the necessary software. Our starting points will look pretty similar no matter the operating system at this point. Since I use windows mostly, my example will be done in that OS, but it really doesn’t matter much at this point.

I have mentioned before Vagrant can use several “providers” to run virtual machines. The reason we’re using Virtualbox is that Vagrant runs with it out of the box and the setup is pretty darn easy.

Before we do anything else, we will install a plugin that will allow providers and Vagrant to talk a bit smoother even if there are version upgrades that reset guest-additions. You can find some information about this here, but for those with blind faith, type this:

vagrant plugin install vagrant-vbguest

I ran this demo on a windows machine so my first step is to start Cygwin, which by default puts me in my working folder C:\tools\cygwin\home\Devinder. This is as good a place as any to work, and it’s completely fine if you use a different folder as a starting point.

Create a new folder:

mkdir vagrant_test

and move into it:

cd vagrant_test

We are going to start with the standard example on the Vagrant official site, which is to initialize a stripped down 12.04 version ubuntu box made by Hashicorp, the vagrant parent company:

vagrant init hashicorp/precise32

Here is what the above commands look like for me:

So Vagrant has generated up ‘Vagrantfile’ for us. This is that script I mentioned earlier that will guide a lot of the configuration for your system. I encourage you to go to your working folder (vagrant_test in my case) whichever way you prefer and open up this ‘Vagrantfile’ in your favorite text editor.  You will see a lot of comments describing different types of commands you can use, and then two uncommented commands that allow the most basic level of configuration

‘Vagrant.configure(“2”) do |config|’

‘config.vm.box = “hashicorp/precise32″‘

These two lines setup the configuration version for the Vagrant API and point to the box we’ll be getting up and running respectively. As the comments will tell you, every Vagrant development requires a box, and you can find different boxes over at  https://atlas.hashicorp.com/search.  Now that we know what’s going on, let’s tell Vagrant to setup everything and download the machine we need for this project if it doesn’t have it handy (which it won’t at this point since we’re running fresh). Type in:

vagrant up

and now let vagrant run its magic. There is a download of a machine in progress so, this can take a few minutes. Once it’s done, we can ssh into the Vagrant machine. Let’s make sure it’s running first:

vagrant status

You should see a reassuring ‘running’ message:

Now let’s ssh into it. If you don’t know what ssh is, we’re forming a secure link with the newly running virtual operating system and ssh (secure shell) is the means to do it on the console. Vagrant boxes will have this set up for us, which is really nice. Also good to note is that the default username/password for the Hashicorp box will be ‘vagrant’/’vagrant’, but you won’t need them when using ssh. Go ahead and type in:

vagrant ssh

You should get a welcome message assuring you that you’re in the virtual machine, but you can double-check by traveling around the file system or typing up:

uname -a

Great, we have managed to initialize a machine spec, and generate a running machine! Before we leave this machine, I want to show you something you get for free with Vagrant, a synced folder. Let’s go to the root /vagrant folder in your ubuntu machine:

cd /vagrant

Now list what’s in here:

ls

You should see, curiously enough, one entry called ‘Vagrantfile’. This is the same exact ‘Vagrantfile’ we saw when we initialized Vagrant on our host machine(Windows). This root /vagrant folder is synced between your machines. Anything you do in the Linux box will show up in the windows folder, and vice-versa. So if you have a favorite text editor in your host machine, that you would like to use in your guest machine, you can do that. You could create a file in this folder, edit and save it in your favorite editor and then use your guest machine to compile or run it. Neat huh? Let’s prove I m not lying.

While still in your ssh session console window, type in:

touch spoon.txt

Now head over to your Host windows machine window where you have your ‘vagrant_test’ folder open, and you should see a ‘spoon.txt’ file there. Open it up in your favorite text editor  and type in something like:

'there is no'

and type enter to get to a new line, then save/exit.

Now head back to your guest machine ssh session and type in:

head spoon.txt

You should see your message in the command prompt.

Since we’re working in a linux guest machine, you might want to ensure that your text editor is configured to work with it. Windows machines will especially have this issue, and if you’re saving files on git down the road, please make sure your account for things like CRLF vs lf and indent styles and sizes. i use an editorconfig plugin for sublime text to prevent any issues. This  won’t be important for this ‘hello world’ section, but it will creep up quickly.

 

For now, we’re done here Let’s exit the ssh session by typing the following in our console:

exit

We’ve exited, but we haven’t really shut down the virtual machine, it’s still running. When you’re back in your Cygwin environment on your host machine, you have the following options to free up resources:

If you want the virtual machine to ‘hibernate’ but to stay on use:

vagrant suspend 

You can reverse this by typing:

vagrant resume

If you want to shut down the VirtualMachine (VM) entirely, you can use:

vagrant halt

You can boot the VM back up again with

vagrant up

If you want to completely destroy the entire VM including everything ever you had on it, use the command:

vagrant destroy

and confirm ‘y’. A quick ‘vagrant status’ check should let you see that the machine is indeed gone for good. Vagrant destroy is pretty useful to clear up space once you’re done experimenting with a machine or if you don’t want to bother with the changes you made and just want a fast reset on the box. You can save a lot of disk space by destroying machines you don’t use. All your files in the synced folder on the host machine will be preserved, so if you used the command to destroy a machine and build it up with a different configuration, you can still use all the files in that folder on the new VM.

There are more commands to learn about, but you now have all the necessary tools to get up and running with your Vagrant machine. You can learn about commands from the Vagrant homepage or you can get some quick help while in the console of your host machine by typing

vagrant -h

At this point, you know everything you really need to use Vagrant on a day-to-day basis. If it’s for the case where you’re using Vagrant to test an OS. You just need to go to the Vagrant site to pick up the connect ‘init’ command for your OS of choice or google around for it, and then you can ‘vagrant up’ a machine and start tinkering.

In the following section, We’ll  discuss a few basic features of Vagrant that make it much more usable. All these features will stem from better manipulation of the Vagrantfile.

Getting more out of Vagrant

The Vagrantfile is essentially a ruby file that controls your VM’s settings. The config.vm in the second line is the namespace where we’ll find most of the parameters we might want to configure. After you’re done making changes to the Vagrantfile, remember to run ‘vagrant reload’ so they come into effect. There are many parameters you can effect, including the hostname, but the first usage I want to cover is provisioning.

Provisioning

Having a vagrant machine you can build is neat, but if you want to install specific software in it every time you spin up the machine, you probably want to automate the process. This is not an uncommon requirement, and in fact, its a process is called “provisioning”. Imagine we want an ubuntu box, preferably a more recent 64-bit Xenial machine and have it pre-installed with python. A quick google search for ‘vagrant ubuntu 64bit 16.04’ and we’re in business. Let’s create the folder and change directories to it.

mkdir vagrant_xenial

cd vagrant_xenial

Then we use the command we found:

vagrant init ubuntu/xenial64 --minimal

This should generate a Vagrantfile we can start working within our working directory. The ‘–minimal’ tag just removes all the comments in the Vagrantfile for us, so it’s easier to work through it. Open up the Vagrantfile in your favorite text editor and add:

config.vm.provision :shell, path: "bootstrap.sh"

This allows you to add a provisioning file of type shell, via a script in your working folder that you’ll call “bootstrap.sh”. This file will contain our instructions for provisioning when we “vagrant up”. Create this file in your working folder, right next to the Vagrantfile:

touch bootstrap.sh

and open it up in an editor as well. Type in the following code into this file:

What we’ve done here is picked up the standard updates from the apt-get package manager, and afterward installed the python tools. We’re about ready to go. Make sure you save both files, and if this is the first time you’ll be starting up this machine, then just go to your console in this folder and type:

vagrant up

If this isn’t going to be the first time you use ‘vagrant up’ for this machine, your provisioning will not go into effect. To trigger provisioning after the first time you run ‘vagrant up’, you need to use the ‘vagrant provision’ command.

To check that everything installed, ssh into your machine and type in

pip

You should see a bunch of information on your pip installation, and you’re all set. You can ‘exit’ this session at this point. Congratulations, you’ve learned some basic provisioning in a shell with Vagrant. You now have the recipe to build a Ubuntu 16.04 box with python tools pre-installed. You can, of course, introduce more complex statements or use third-party tools to do this, but what you’ve done should give you some confidence to take further steps.

Networking

You can forward ports from your VM to your host machine. This is useful if you’re running a server on your VM, and want to see how changes you make to files affect the page in the browser. Open up your Vagrantfile and add the following line at the bottom of what you already have:

config.vm.network :forwarded_port, guest: 80, host: 4567, auto_correct: true

The network configuration above allows for forwarding ports from a guest machine to the host machine as specified. The auto_correct option resolved collisions with ports that are already bound and automatically reassigns ports for you. You can specify connection types, label them and do more fine-grained networking changes, but the command we’re using will serve our purposes.

If you had a server running on port 80 in your guest machine, you would be able to check it out on ‘localhost:4567’ on your browser. It’s lame to just say that and not demonstrate it, so lets set something up quickly. Open up your ‘bootstrap.sh’ file and add the following lines:

apt-get -y install nginx

to install the Nginx server, and:

service nginx start

to get the service started after the install. Here is what the files should look like at this point:

The Vagrantfile:

The bootstrap.sh:

Save, exit, go over to the console, and run:

vagrant reload

so that our changes to the Vagrantfile are recognized, and then:

vagrant provision

so that our provision file gets run again. Now let’s go over to our web-browser in the host operating system and type in:

localhost:4567

You should see a welcome message from Nginx that looks something like this:

Congratulations, You can at least begin to set up a server on Vagrant. One last trick I’ll mention is about the shared folders:

Changing Synced folders

By default, your entire working folder in the Host machine is shared with the virtual machine in the ‘/vagrant’ folder. You can change this with a few simple additions in the Vagrantfile. This is useful if your working folder is part of a repository that has other files that you don’t want to expose to your guest machine. Let’s demonstrate how this would work. Open up the Vagrantfile we’ve been working on, and type in:

config.vm.synced_folder "./", "/vagrant", disabled: true

The above code allows you to remove the sharing between the entire host working folder and the ‘/vagrant’ folder in the guest machine.

config.vm.synced_folder “my_host_folder”, “/vagrant/my_guest_folder”

For this to work, you need to create the ‘my_host_folder’ in your working folder on the host machine inside the working folder. the ‘my_guest_folder’ on the guest machine will be made automatically. Then after saving and exiting the Vagrantfile, you’ll need to put the new settings in the Vagrantfile in effect:

vagrant reload

There you have it, you should now be only sharing the ‘my_host_folder’ on your host machine working folder, with the guest machine. The same shared folder in the guest machine would be called ‘my_guest_folder’. Normally I wouldn’t advise creating different folder names with the same content like this, but for the purposes of demonstration, it makes things super clear.

Given what you’ve learned in this article, you should have more confidence to roam through Vagrant articles and browse through Vagrant configurations.  I have a few recommendations I can make if you want to continue your learning about the subject.

Where to go from here:

  • To get a sense of Vagrant and its philosophy, you might want to check out Mitchell Hashimoto’s website. He is the founder of Hashicorp and is behind the creation of the software.
  • Vagrant comes with a “share” command that I didn’t get a chance to introduce in this article. You can essentially share your VM over the internet if you have a free Hashicorp account. This is a good opportunity to make use of the official documentation.
  • In this article, I used the precise32 box by Hashicorp, but I don’t want you thinking only Hashicorp makes boxes. Many organizations and individuals make boxes for Vagrant and it’s entirely possible to make your own. There is step by step instruction over at SitePoint.
  • Vagrant can make use of provisioning tools like Chef or Puppet or Ansible to simplify setup and I would recommend looking into these options. I came across an article over at scotch.io that might get you started.
  • One good way to get experience with Vagrant is to use projects that make use of it. Github has several use cases of Vagrant, here are a couple I’ve actually made use of:
    • Trellis: This is a WordPress Lemp Stack that makes use of Vagrant and Ansible playbooks. There is a bit of a learning curve, but super useful once you understand what it’s doing.
    • Tensorflow-ipy: I heard about Tensorflow from my friend Dobri and wanted to get up and running quickly, this little project let me do exactly that.

Credits:

This tutorial comes as a condensation of a lot of combined knowledge. I use a lot of the regular internet resources including Google, Wikipedia, and StackOverflow. Still, credit is due here:

  1. Professor Frees at Ramapo College introduced me to Vagrant.
  2. The official documentation at Vagrant was source material for a lot of this tutorial and deserves mention.
  3. Wes Higbee’s tutorial on Vagrant and versioning on Pluralsight was a source of inspiration, especially for getting things running on windows with better organization.
  4. Everyone who helped me proof this very first article, on this very new website. (Please feel free to share any comments you might have about it).

That is all folks!

I started programming at school around 3 years ago. There’s a lot to learn and share about and I use my posts as a medium for that.

dev_sodhi

I started programming at school around 3 years ago. There's a lot to learn and share about and I use my posts as a medium for that.

2 thoughts on “Smarter project configuration with Vagrant

  1. Very nicely done and informative! One note though – it seems vagrant has some pretty decent UI support and UI-enabled images but this is a bit focused on command line interactions, even where it’s a bit awkward – under Windows. What was your reasoning behind that? It seems that taking advantage of attaching UI to your images can help you use Vagrant as a combination of both image and development environment management software rather than just another image manager.

    1. Thanks for the compliment and the questions! Vagrant supports a UI if you enable it in the Vagrantfile, but since this is not how you would use a Linux machine generally- and I chose Linux guest machines to demonstrate, it made sense to use the standard. I say standard because that’s how I’ve seen Linux used, pretty much always. This includes tools that integrate Linux machines like Trellis, the cloud interface you’d expect to see from amazon or digital ocean for a Linux machine that might even serve as a starting point for tools like Docker. Considering the nature of other tutorials that I referenced, and how I use it generally – I had a bias towards using a command line interface. Vagrant remains flexible as a CLI tool regardless of the host operating system. There are exceptions to this generalization. If I used a guest Windows machine, then it would make more sense to use the UI, because windows naturally plays well with UI as Linux does CLI.

Leave a Reply