Vagrant CoreOS EC2 and VirtualBox

Okay in a previous post I had been looking at Vagrant. I have a Vagrant file I liked to bring up ubuntu on either VirtualBox or AWS. But my target isn't really ubuntu (well not on the metal anyway.) I was looking to install CoreOS and then I will layer ubuntu on top of it. So how to get Vagrant to provision a CoreOS box. So lets see what Dr. Google says. https://coreos.com/docs/running-coreos/platforms/vagrant/

Hokay looks like they want me to get a Vagrant file.

git clone https://github.com/coreos/coreos-vagrant.git
cd coreos-vagrant

Let's take a peek at what we have.

$ ls -la
total 80
drwxr-xr-x  11 jahnke  staff   374 Feb 23 20:25 ./
drwxr-xr-x   3 jahnke  staff   102 Feb 23 20:25 ../
drwxr-xr-x  13 jahnke  staff   442 Feb 24 20:27 .git/
-rw-r--r--   1 jahnke  staff   117 Feb 23 20:25 .gitattributes
-rw-r--r--   1 jahnke  staff    35 Feb 23 20:25 .gitignore
-rw-r--r--   1 jahnke  staff  2448 Feb 23 20:25 CONTRIBUTING.md
-rw-r--r--   1 jahnke  staff   104 Feb 23 20:25 MAINTAINERS
-rw-r--r--   1 jahnke  staff  4240 Feb 23 20:25 README.md
-rw-r--r--   1 jahnke  staff  4355 Feb 23 20:25 Vagrantfile
-rw-r--r--   1 jahnke  staff  2353 Feb 23 20:25 config.rb.sample
-rw-r--r--   1 jahnke  staff   722 Feb 23 20:25 user-data.sample

Interesting... There is a Vagrantfile and 2 'other' files. So lets peek in the Vagrant file first. Whoa, it is 135 lines long. Holy crap let me dig into this.

require 'fileutils'

Vagrant.require_version ">= 1.6.0"

CLOUD_CONFIG_PATH = File.join(File.dirname(__FILE__), "user-data")
CONFIG = File.join(File.dirname(__FILE__), "config.rb")

# Defaults for config options defined in CONFIG
$num_instances = 1
$instance_name_prefix = "core"
$update_channel = "alpha"
$enable_serial_logging = false
$share_home = false
$vm_gui = false
$vm_memory = 1024
$vm_cpus = 1
$shared_folders = {}

This first bit defines the two 'config' files we saw. As well as sets the default vars. These values can be overridden in 'config.rb' file. So I only need one Vagrant file that can be reused by changing the config.rb file. I mean the Vagrant file exists so I can have a Vagrant file per install why this extra flexibility is needed is beyond me. But what do I know? what I will do is change the user-data and config.rb to 'vagrant-user-data' and 'vagrant-config.rb' so that it is more obvious why these files are hanging out in my app dir.

Okay next bit.

# Attempt to apply the deprecated environment variable NUM_INSTANCES to
# $num_instances while allowing config.rb to override it
if ENV["NUM_INSTANCES"].to_i > 0 && ENV["NUM_INSTANCES"]
  $num_instances = ENV["NUM_INSTANCES"].to_i
end

if File.exist?(CONFIG)
  require CONFIG
end

# Use old vb_xxx config variables when set
def vm_gui
  $vb_gui.nil? ? $vm_gui : $vb_gui
end

def vm_memory
  $vb_memory.nil? ? $vm_memory : $vb_memory
end

def vm_cpus
  $vb_cpus.nil? ? $vm_cpus : $vb_cpus
end

It looks like at some point in the past there was another way to do this. So some people might be setting NUM_INSTANCES as a shell var. And other configs might have vb_ (for virtual box) vars. The only bit of this that I will keep is thing that loads in the CONFIG file. The rest isn't needed (although we have to make a small change later in the script to accommodate the vm_ vars.

Vagrant.configure("2") do |config|
  # always use Vagrants insecure key
  config.ssh.insert_key = false

  config.vm.box = "coreos-%s" % $update_channel
  config.vm.box_version = ">= 308.0.1"
  config.vm.box_url = "http://%s.release.core-os.net/amd64-usr/current/coreos_production_vagrant.json" % $update_channel

  ["vmware_fusion", "vmware_workstation"].each do |vmware|
    config.vm.provider vmware do |v, override|
      override.vm.box_url = "http://%s.release.core-os.net/amd64-usr/current/coreos_production_vagrant_vmware_fusion.json" % $update_channel
    end
  end

  config.vm.provider :virtualbox do |v|
    # On VirtualBox, we don't have guest additions or a functional vboxsf
    # in CoreOS, so tell Vagrant that so it can be smarter.
    v.check_guest_additions = false
    v.functional_vboxsf     = false
  end

  # plugin conflict
  if Vagrant.has_plugin?("vagrant-vbguest") then
    config.vbguest.auto_update = false
  end

This is looking pretty normal for a Vagrant file (well the ones I have seen at any rate.) We are setting the box name and some other info that will be used to grab the correct version of the box. Then some specific settings for both vmware as well as virtual box. And finally disable a conflicting plugin. Since I don't run vmware on my box, and I looked at the vaguest plugin and it seemed kind of out of date and not at all interesting to me I won't be using it either. Both of these segments can go.

This next bit was a little interesting.

  (1..$num_instances).each do |i|
    config.vm.define vm_name = "%s-%02d" % [$instance_name_prefix, i] do |config|
      config.vm.hostname = vm_name
 

What appears to be happening here is the Vagrant file is making different config sets for each machine. In the config.rb file you can define how many you want to start. And this is creating a config for each and naming it by adding a 2 digit number to the end of the name. Sweet...

      if $enable_serial_logging
        logdir = File.join(File.dirname(__FILE__), "log")
        FileUtils.mkdir_p(logdir)

        serialFile = File.join(logdir, "%s-serial.txt" % vm_name)
        FileUtils.touch(serialFile)

        ["vmware_fusion", "vmware_workstation"].each do |vmware|
          config.vm.provider vmware do |v, override|
            v.vmx["serial0.present"] = "TRUE"
            v.vmx["serial0.fileType"] = "file"
            v.vmx["serial0.fileName"] = serialFile
            v.vmx["serial0.tryNoRxLoss"] = "FALSE"
          end
        end

        config.vm.provider :virtualbox do |vb, override|
          vb.customize ["modifyvm", :id, "--uart1", "0x3F8", "4"]
          vb.customize ["modifyvm", :id, "--uartmode1", serialFile]
        end
      end

This bit turns on the serial logging for the VM. Once again I don't use VM Ware, I can't use it for AWS and it looks like it is expensive for virtual box. I could probably do without any of it. But I will leave just the non VM ware bits in place.

      if $expose_docker_tcp
        config.vm.network "forwarded_port", guest: 2375, host: ($expose_docker_tcp + i - 1), auto_correct: true
      end

This isn't useful for AWS either, but it will work for VirtualBox.

      ["vmware_fusion", "vmware_workstation"].each do |vmware|
        config.vm.provider vmware do |v|
          v.gui = vm_gui
          v.vmx['memsize'] = vm_memory
          v.vmx['numvcpus'] = vm_cpus
        end
      end

      config.vm.provider :virtualbox do |vb|
        vb.gui = vm_gui
        vb.memory = vm_memory
        vb.cpus = vm_cpus
      end

So remember the bits up top I was thinking about getting rid of? They will have to be fixed here. Adding a $ in front of vm_ will fix them. And get rid of the vmware bits.

      ip = "172.17.8.#{i+100}"
      config.vm.network :private_network, ip: ip

Since we are starting more than one box, this will increment the VirtualBox ip addresses, AWS will of course ignore this totally.

Finally

      # Uncomment below to enable NFS for sharing the host machine into the coreos-vagrant VM.
      #config.vm.synced_folder ".", "/home/core/share", id: "core", :nfs => true, :mount_options => ['nolock,vers=3,udp']
      $shared_folders.each_with_index do |(host_folder, guest_folder), index|
        config.vm.synced_folder host_folder.to_s, guest_folder.to_s, id: "core-share%02d" % index, nfs: true, mount_options: ['nolock,vers=3,udp']
      end

      if $share_home
        config.vm.synced_folder ENV['HOME'], ENV['HOME'], id: "home", :nfs => true, :mount_options => ['nolock,vers=3,udp']
      end

      if File.exist?(CLOUD_CONFIG_PATH)
        config.vm.provision :file, :source => "#{CLOUD_CONFIG_PATH}", :destination => "/tmp/vagrantfile-user-data"
        config.vm.provision :shell, :inline => "mv /tmp/vagrantfile-user-data /var/lib/coreos-vagrant/", :privileged => true
      end

    end
  end
end

The rest of this is to deal with sharing the file systems between VirtualBox and the host system. I don't actually use this right now. But I will leave it in place incase I find a compelling reason to use it.

If you remember earlier I needed to set some values to make AWS work correctly. As it turns out the config bits are a little bit tricky if I set the config.vm.box in one place it won't be able to be reset in a different place. And both the VirtualBox and AWS use different boxes. So some reading up on how the Ruby do iterator was working led me to the following answer.

  config.vm.provider :virtualbox do |v, override|
    override.vm.box = "coreos-%s" % $update_channel
    override.vm.box_version = ">= 308.0.1"
    override.vm.box_url = "http://%s.release.core-os.net/amd64-usr/current/coreos_production_vagrant.json" % $update_channel
    # On VirtualBox, we don't have guest additions or a functional vboxsf
    # in CoreOS, so tell Vagrant that so it can be smarter.
    v.check_guest_additions = false
    v.functional_vboxsf     = false
  end

  config.vm.provider :aws do |aws, override|
      aws.access_key_id = ENV['AWS_ACCESS_KEY_ID']
      aws.secret_access_key = ENV['AWS_SECRET_ACCESS_KEY']
      aws.keypair_name = "jahnke-gsg-keypair"
      aws.ami = "ami-8097d4e8"
      aws.region = "us-east-1"
      aws.instance_type = "t1.micro"

      override.vm.box = "dummy"
      override.ssh.username = "core"
      override.ssh.private_key_path = "/Users/jahnke/ec2-keys/jahnke-gsg-keypair.pem"
  end

This way neither config sets it directly they set via the override. And so there won't be a conflict. I statically chose the stable coreOS I need to add a conditional here to choose the right AMI based on the update_channel var. And instead of using ubuntu as the base identity CoreOS uses core. This small addition now lets me start with either VirtualBox or AWS.

require 'fileutils'

Vagrant.require_version ">= 1.6.0"

CLOUD_CONFIG_PATH = File.join(File.dirname(__FILE__), "vagrant-user-data")
CONFIG = File.join(File.dirname(__FILE__), "vagrant-config.rb")

# Defaults for config options defined in CONFIG
$num_instances = 1
$instance_name_prefix = "core"
$update_channel = "alpha"
$enable_serial_logging = false
$share_home = false
$vm_gui = false
$vm_memory = 1024
$vm_cpus = 1
$shared_folders = {}

if File.exist?(CONFIG)
  require CONFIG
end

Vagrant.configure("2") do |config|
  # always use Vagrants insecure key
  config.ssh.insert_key = false

  config.vm.provider :virtualbox do |v, override|
    override.vm.box = "coreos-%s" % $update_channel
    override.vm.box_version = ">= 308.0.1"
    override.vm.box_url = "http://%s.release.core-os.net/amd64-usr/current/coreos_production_vagrant.json" % $update_channel
    # On VirtualBox, we don't have guest additions or a functional vboxsf
    # in CoreOS, so tell Vagrant that so it can be smarter.
    v.check_guest_additions = false
    v.functional_vboxsf     = false
  end

  config.vm.provider :aws do |aws, override|
      aws.access_key_id = ENV['AWS_ACCESS_KEY_ID']
      aws.secret_access_key = ENV['AWS_SECRET_ACCESS_KEY']
      aws.keypair_name = "jahnke-gsg-keypair"
      aws.ami = "ami-8097d4e8"
      aws.region = "us-east-1"
      aws.instance_type = "t1.micro"

      override.vm.box = "dummy"
      override.ssh.username = "core"
      override.ssh.private_key_path = "/Users/jahnke/ec2-keys/jahnke-gsg-keypair.pem"
  end

  (1..$num_instances).each do |i|
    config.vm.define vm_name = "%s-%02d" % [$instance_name_prefix, i] do |config|
      config.vm.hostname = vm_name

      if $enable_serial_logging
        logdir = File.join(File.dirname(__FILE__), "log")
        FileUtils.mkdir_p(logdir)

        serialFile = File.join(logdir, "%s-serial.txt" % vm_name)
        FileUtils.touch(serialFile)

        config.vm.provider :virtualbox do |vb, override|
          vb.customize ["modifyvm", :id, "--uart1", "0x3F8", "4"]
          vb.customize ["modifyvm", :id, "--uartmode1", serialFile]
        end
      end

      if $expose_docker_tcp
        config.vm.network "forwarded_port", guest: 2375, host: ($expose_docker_tcp + i - 1), auto_correct: true
      end

      ip = "172.17.8.#{i+100}"
      config.vm.network :private_network, ip: ip

      config.vm.provider :virtualbox do |vb|
        vb.gui = $vm_gui
        vb.memory = $vm_memory
        vb.cpus = $vm_cpus
      end

      # Uncomment below to enable NFS for sharing the host machine into the coreos-vagrant VM.
      #config.vm.synced_folder ".", "/home/core/share", id: "core", :nfs => true, :mount_options => ['nolock,vers=3,udp']
      $shared_folders.each_with_index do |(host_folder, guest_folder), index|
        config.vm.synced_folder host_folder.to_s, guest_folder.to_s, id: "core-share%02d" % index, nfs: true, mount_options: ['nolock,vers=3,udp']
      end

      if $share_home
        config.vm.synced_folder ENV['HOME'], ENV['HOME'], id: "home", :nfs => true, :mount_options => ['nolock,vers=3,udp']
      end

      if File.exist?(CLOUD_CONFIG_PATH)
        config.vm.provision :file, :source => "#{CLOUD_CONFIG_PATH}", :destination => "/tmp/vagrantfile-user-data"
        config.vm.provision :shell, :inline => "mv /tmp/vagrantfile-user-data /var/lib/coreos-vagrant/", :privileged => true
      end

    end
  end
end

My clevah blogging platform

So I want to add a blog to my website http://www.tormenta.com/ but I don't want to actually build a complete blogging platform nor do I want to put something like wordpress on my site.

So some head scratching later I realized what I wanted to do is use someone else's platform and just put the words up on my site. I don't actually care if people can interact with me on my site at all (well not today so I am not going to plan for it.) I was looking for something that worked with blogging tools (like MarsEdit which is what I use) but displayed on my website. Then I realized that what I could do is put up a blog somewhere and use MarsEdit to work on it there and then have it synced back to my site via the Goolge API.

Like all ideas of this nature it seems a little stupid but I think it gives me most of what I want. I can include the original URL for people to get RSS feeds or if I want to hook it to twitter or some such. And really I don't need to write all that much software. Just the bits that collect the data and display it on my site (add a few tools to do forward and back and a history.)

So lets get started. First thing I need is some code to get the API data for me. And nom comes to the rescue, although technically I don't NEED this it will make life a little bit easier. https://github.com/google/google-api-nodejs-client/. Next I need an API key for my app. I need to ask Google for permission to use their blogger API. First step is go to https://console.developers.google.com/project and make a project if you don't already have one. Once you have a project then go to the APIs & auth and choose the API you want to enable. In my case it was blogger v3. And click on the "ON" button. They will ask you what you want the API for and ask you how much you will need. I clicked on the button and am now waiting for a response. Once I get it I will continue.

Vagrant and EC2

I needed to get this up and running today. Well perhaps needed wasn't exactly the right words. But here is what I did.

First off because I have a bad case of Programmer ADD I wanted to fix some of my identity issues with EC2. So I went created a new group called devops with EC2 and S3 Full entitlements on it. And then added an account with a password and got a new ID and Key. Then I deactivated the root ID and Key (I did not delete as I might find something that needs it later.) While I was there I protected my root key with MFA. Which should reduce the likelihood of bad guys stealing from me.

Next I have a set of scripts that execute when I bring up a shell and set env vars for my ID and Key (which is perhaps not the smartest move but this from my laptop and desktop both are single user computers, so I doubt it is that bad.) Now that I have my user and key I went to look for vagrant and aws. And what I found was this.

https://github.com/mitchellh/vagrant-aws

Sweet a lot of what I needed was there. So first thing I need to do is install it.

$ vagrant plugin install vagrant-aws
...
$ vagrant up --provider=aws
...

Aaaaaaaand it didn't work. It was not able to find the box. So I had to keep reading. A 'dummy' box needs to be configured. So unlike regular vagrant which copies stuff to your machine and starts a VM there. This was just reaching out into the API for EC2.

$ vagrant box add dummy https://github.com/mitchellh/vagrant-aws/raw/master/dummy.box
...

Near as I can tell all this does is installed this file. It might do more but I am as yet unfamiliar with the ways of Vagrant.

$ cat ~/.vagrant.d/boxes/dummy/0/aws/metadata.json 
{
    "provider": "aws"
}

I tried again and it didn't work again. What it wanted was a Vagrant file. Which is OK so I copied their sample.

Vagrant.configure("2") do |config|
  config.vm.box = "dummy"

  config.vm.provider :aws do |aws, override|
    aws.access_key_id = "YOUR KEY"
    aws.secret_access_key = "YOUR SECRET KEY"
    aws.session_token = "SESSION TOKEN"
    aws.keypair_name = "KEYPAIR NAME"

    aws.ami = "ami-7747d01e"

    override.ssh.username = "ubuntu"
    override.ssh.private_key_path = "PATH TO YOUR PRIVATE KEY"
  end
end

So the first thing that threw me was session_token? What the hell was that? I looked around for a bit and I learned more about session tokens but I still don't understand them. So I wondered if I could just not include them and would things still work? And as it turns out they do. Sweet....

But now, how am I gonna check this Vagrant file in? It will have my access and secret in it (committing passwords is a really stupid idea and one should not do that. So I needed a plan B. And I remembered two things. First was that I had the ID and Secret as env variables, so that was nice. And the second thing was that Vagrant files are Ruby Files. So some searching and I learned that ENV is a list of the environment variables. Soo lets see if this works.

Vagrant.configure("2") do |config|
  config.vm.box = "dummy"

  config.vm.provider :aws do |aws, override|
    aws.access_key_id = ENV['AWS_ACCESS_KEY_ID']
    aws.secret_access_key = ENV['AWS_SECRET_ACCESS_KEY']
    aws.keypair_name = "jahnke-gsg-keypair"

    aws.ami = "ami-86aa1cee"

    override.ssh.username = "ubuntu"
    override.ssh.private_key_path = "/Users/jahnke/ec2-keys/jahnke-gsg-keypair.pem"
  end
end

And one more time and yes it did work. It brought one up in my default Zone (which is VA.) This is kinda sweet, I could ssh into it and everything. I currently use a script called getawsaddresses which helps me find what machines I want.

$ getawsaddresses 
 1 - ubuntu/images/ebs-ssd/ubuntu-trusty-14.04-i386-server-20140927 2014-11-10T15:37:05.000Z
     ec2-54-253-633-89.compute-1.amazonaws.com
     ssh -i /Users/jahnke/ec2-keys/jahnke-gsg-keypair.pem ubuntu@ec2-54-253-633-89.compute-1.amazonaws.com
Choose a machine: 

So this is going to be used to build my main tormenta.com web server. What I want to do is learn how to build a machine with CoreOS and use Docker to deploy. So I need really for this to run a few different ways. One in AWS so I can deploy and easily but also I want to run this in a local VM so I can experiment. So I came up with the following Vagrant file.

Vagrant.configure("2") do |config|
  config.vm.define :tormenta do |x|
    x.vm.box = "hashicorp/precise64"
    x.vm.hostname = "tormenta"

    x.vm.provider :virtualbox do |v|
      v.name = "tormenta"
    end

    x.vm.provider :aws do |aws, override|
      aws.access_key_id = ENV['AWS_ACCESS_KEY_ID']
      aws.secret_access_key = ENV['AWS_SECRET_ACCESS_KEY']
      aws.keypair_name = "jahnke-gsg-keypair"
      aws.ami = "ami-86aa1cee"
      aws.region = "us-east-1"
      aws.instance_type = "t1.micro"

      override.vm.box = "dummy"
      override.ssh.username = "ubuntu"
      override.ssh.private_key_path = "/Users/jahnke/ec2-keys/jahnke-gsg-keypair.pem"
    end
  end
end

Aaaaaaand that worked a treat. It brought the machine up and Vagrant knows what machine it is and everything. I have no trouble imagining that I can do this and it will keep track of it between reboots. And it shows up on my existing tool to look at active EC2 instances.

$ getawsaddresses 
 1 - ubuntu/images/ebs-ssd/ubuntu-trusty-14.04-i386-server-20140927 2014-11-10T15:37:05.000Z
     ec2-54-253-633-89.compute-1.amazonaws.com
     ssh -i /Users/jahnke/ec2-keys/jahnke-gsg-keypair.pem ubuntu@ec2-54-253-633-89.compute-1.amazonaws.com
 2 - ubuntu/images/ebs-ssd/ubuntu-trusty-14.04-i386-server-20140927 2015-02-23T02:33:29.000Z
     ec2-54-205-188-112.compute-1.amazonaws.com
     ssh -i /Users/jahnke/ec2-keys/jahnke-gsg-keypair.pem ubuntu@ec2-54-205-188-112.compute-1.amazonaws.com
Choose a machine: 

So what's next? I need to get security groups to work. So I can lock the machine down to do only what I want it to do. And then I need to get it to boot into CoreOS and not Ubuntu. And finally build some containers with the web components and have them built the machine. But as of right ow the Vagrant file works file and I will be incorporating it into the source of the web app its self.