Getting Started with Vagrant

Vagrant is a virtualization technology that allows you to configure virtualization software such as Linux Containers and VirtualBox. It is commonly used together with orchestration tools like Ansible, and Chef.

To get started, download Vagrant here - https://www.vagrantup.com/downloads.html

Create a folder and run

vagrant init

This should create a VagrantFile.

Similar to Docker, Vagrant is dependent on base images. Let's begin by downloading an ubuntu box:

vagrant box add hashicorp/precise32

Open VagrantFile and edit the following:

config.vm.box = "hashicorp/precise32"

You can always find other boxes here:

https://atlas.hashicorp.com/boxes/search

Let's boot up the box:

vagrant up
vagrant ssh

You can check the status of the machine by running:

vagrant status

Do not delete the folder /vagrant, it's a synced folder

Let's begin by loading a script that will install apache.

In your hosts machine's root folder, create s file called bootstrap.sh

vi bootstrap.sh

Add the following:

apt-get update
apt-get install -y apache2
if ! [ -L /var/www ]; then
  rm -rf /var/www
  ln -fs /vagrant /var/www
fi

In VagrantFile, add

Vagrant.configure("2") do |config|
  config.vm.box = "hashicorp/precise32"
  config.vm.provision :shell, path: "bootstrap.sh"
end

Reload the provision:

vagrant reload --provision

Test the status of apache:

vagrant ssh
service apache2 status

Try running:

wget -qO- 127.0.0.1

Add the following line in VagrantFile for port forwarding, so we can see webpages from our host browsers

config.vm.network :forwarded_port, guest: 80, host: 4567

Run

vagrant reload

In your browser, do

http://127.0.0.1:4567

If you want to share this image to Altas to share/backup your files, register an account at

https://atlas.hashicorp.com/

Run

vagrant login
vagrant share

In the browser, access the url that's outputted by the terminal.

When you finished sharing, Ctrl = C to terminate it.

When you are done with your vagrant box, you can use the following:

vagrant suspend - state is saved, quick to start up, consumes space
vagrant halt - guest OS is shut down, consumes space
vagrant destroy - removes the guest machine

You can use vagrant up to start it again.

By default, the vagrant box is backed with Virtual Box.

But you can easily change it to VMware or AWS by:

vagrant up --provider-vmware_fusion
vagrant up --provider=aws

Migrating Splunk indexed data

First stop splunk.

cd into your splunk/bin directory
./splunk stop

Create a new folder (ex. /mnt/splunk_data).

cp -rp splunk/var/lib/splunk/* /mnt/splunk_data/

Change SPLUNK_DB to point to /mnt/splunk_data.

vi splunk/etc/splunk-launch.conf

Find SPLUNK_DB in the file and change the path.

SPLUNK_DB=/mnt1/splunk_data

You may also want to change the retention policy and the max storage size.

// 30 days
frozenTimePeriodInSecs = 2592000

// 90G
maxTotalDataSizeMB = 90000

It's recommended to set the size using the following formula:

Total storage = daily average rate x retention policy x 1/2 = 15 Gig

Start Splunk.

./splunk start

To tune Splunk settings, check:
http://docs.splunk.com/Documentation/Splunk/4.3.1/Installation/CapacityplanningforalargerSplunkdeployment

Install Splunk Forwarding and Receiving

We will be using Splunk Light.

Click on the menu icon at the upper right corner. Choose Data -> Receiving.

In Configure receiving, choose 9997 as the receiving port.

In your application instance, install the universal splunk forwarder.

http://www.splunk.com/en_us/download/universal-forwarder.html

Extract it and put it in /opt/splunk_forwarder directory

sudo ./splunk start
sudo ./splunk enable boot-start -user ec2-user

List all the forward servers:
./splunk list forward-server

Active forwards:
None
Configured but inactive forwards:
None

If it prompts you for username and password, use
username: admin
password: changeme

Add the receiving server to the forwarder:

./splunk add forward-server :9997

Test the connection:
./splunk list forward-server

Active forwards:
None
Configured but inactive forwards:
:9997

If it's not active, remember to add port 9997 to your security group.

Add data to monitor

./splunk add monitor -index main -sourcetype

To list what's being monitored:

./splunk list monitor

Installing splunk on AWS

Begin by downloading Splunk Light here: http://www.splunk.com/en_us/download.html. You will probably need to register an account on Splunk before it lets you to download it.

Upload Splunk to your ec2 instance using SCP. For example

scp -i ec2-user@:tmp

In above, I uploaded the splunk tgz file to a tmp folder in my ec2 instance.

You will need to install glibc.i686 first.

yum -y install glibc.i686

Create a folder called /opt if it doesn't exist

Extract your tgz file inside opt

tar xvzf splunklight-6.2.4-271043-Linux-i686.tgz

The splunk executable is located in /opt/splunk/bin. cd into it.

Start splunk:

sudo ./splunk start --accept-license

Start splunk on boot:

sudo ./splunk enable boot-start -user ec2-user

You should be able to view splunk's web interface at port 8000 or your ec2 public address.

Other useful commands:

./splunk stop
./splunk restart

show user cronjobs in ubuntu

Show all the users and their respective cronjobs

for user in $(cut -f1 -d: /etc/passwd); do echo $user; crontab -u $user -l; done

boot2docker cannot cd into a directory

Let's say you are running your server using:

docker-compose up

You may be trying to run bash for your container,

docker ps (grab the container id)
docker exec -it 301 bash

When you cd into a mounted host volume, if you get a "killed" message or it just logs you out, try the following:

boot2docker restart

docker - error fetching ubuntu packages

If you ever see the following error and you are using boot2docker, run "boot2docker restart"

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/libe/libevent/libevent-2.0-5_2.0.21-stable-1ubuntu1.14.04.1_amd64.deb  Could not resolve 'archive.ubuntu.com'

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/m/memcached/memcached_1.4.14-0ubuntu9_amd64.deb  Could not resolve 'archive.ubuntu.com'

E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?