Showing posts with label filesystem. Show all posts
Showing posts with label filesystem. Show all posts

Sunday, May 1, 2016

Xenial Ansible

I started building out some Ubuntu 16.04 (Xenial Xerus) servers this weekend with Ansible, and was impressed by how smoothly it went. The only major issue I encountered was that Ansible requires Python 2.x, whereas Ubuntu 16.04 ships Python 3.5 by default. Fortunately, it's not too hard to work around; here's how I fixed that — and a couple of other issues specific to the servers I was building out:

Python 2

Since Ansible doesn't work with Python 3, and that's what Ubuntu 16.04 provides by default, this is the error I got when I tried running Ansible against a newly-booted server:

/usr/bin/python: not found

So I had to make this the very first Ansible play (bootstrapping the ability of Ansible to use Python 2 for the rest of its tasks, as well as for its other record keeping — like gathering facts about the server):

- name: bootstrap python 2
  gather_facts: no
  tasks:
  - raw: sudo apt-get update -qq && sudo apt-get install -qq python2.7

And in the inventory variables (or group variables) for the server, I had to add this line (directing it to use Python 2 instead of the server's default Python):

ansible_python_interpreter: /usr/bin/python2.7

Aptitude

The next hiccup I ran into was using the Ansible apt module with the upgrade=full option. This option is implemented by using the aptitude program — which Ubuntu no longer installs by default. I was getting this error trying to use that option:

Could not find aptitude. Please ensure it is installed.

So I just tweaked my playbook to install the aptitude package first before running apt: upgrade=full:

- name: install apt requirements
  become: yes
  apt: pkg=aptitude

- name: update pre-installed packages
  become: yes
  apt: upgrade=full update_cache=yes

Mount with nobootwait

Then I started running into some minor issues that were completely unrelated to Ansible — simply changes Ubuntu had picked up between 15.10 and 16.04. The first of these was the nobootwait option for mountall (eg in /etc/fstab mountpoints). This option seems to be no longer supported — the server hung after rebooting, with this message in the syslog:

Unrecognized mount option "nobootwait" or missing value

Maybe this is just an issue with AWS EC2 instance-store volumes, but I had to change the /etc/fstab definition for the server's instance-store volume from this:

/dev/xvdb /mnt auto defaults,noatime,data=writeback,nobootwait,comment=cloudconfig 0 2

To this:

/dev/xvdb /mnt auto defaults,noatime,data=writeback,comment=cloudconfig 0 2

Java 7

The Ubuntu 16.04 repo no longer includes Java 6 or 7 — only Java 8 and 9. I got this error message trying to install Java 7:

No package matching 'openjdk-7-jdk' is available

So I first had to add a PPA for OpenJDK 7, and then could install it:

- name: register java 7 ppa
  become: yes
  apt_repository: repo=ppa:openjdk-r/ppa

- name: install java 7
  become: yes
  apt: pkg=openjdk-7-jdk

But the PPA doesn't include the timezone database, so the time formatting in our app was restricted to GMT. So I had to "borrow" the timezone data from Ubuntu 14.04:

- name: download tzdata-java
  get_url:
    url: http://mirrors.kernel.org/ubuntu/pool/main/t/tzdata/tzdata-java_2016d-0ubuntu0.14.04_all.deb
    dest: ~/tzdata-java.deb
    checksum: sha256:5131aa5219739ac58c00e18e8c9d8c5d6c63fc87236c9b5f314f7d06b46b79fb

- name: install tzdata-java
  become: yes
  command: dpkg --ignore-depends=tzdata -i tzdata-java.deb

That's probably going to be broken in a month or two after the next tzdata update, but it's good enough for now. Later I'll put together some Ansible tasks to build the data from source (although obviously the long-term solution is to upgrade to java 8).

Update 5/14/2016: I created a Xenial tzdata-java package to simplify the installation of Java 7 timezone data.

MySQL root

The final issue I hit was with the MySQL root user. Unlike previous versions of Ubuntu, which by default would come with 3 or 4 MySQL root users (covering all the different variations for naming your local host), and all with empty passwords, Ubuntu 16.04 comes with just a single root user — with which you can login only via MySQL's auth_socket plugin. So I was getting this error trying to login to MySQL as root the old way:

unable to connect to database, check login_user and login_password are correct or /home/me/.my.cnf has the credentials

The new way is simply to login to MySQL using the system's root user (ie sudo mysql). In Ansible tasks (such as those using the mysql_db module), this just means using the become (aka sudo) directive, and omitting the login_user and login_password options:

- name: create new database
  become: yes
  mysql_db: name=thedata

Sunday, November 1, 2015

Sandboxing Firefox with Firejail

I typically run about five or six different instances of Firefox with different profiles for different tasks or groups of websites (like one for dev work, one for my "daily driver", one for financial accounts, one for each organization I work for, etc). I've started using Firejail to better isolate each instance — not only from each other, but from the rest of my system.

Separate Home Directories

To run each profile with a separate home directory, I first created a new ~/fj dir, with a separate directory in it for each Firefox profile (like ~/fj/ff-dev, ~/fj/ff-company-x, etc). Then I moved the existing profile for each into its own .mozilla/firefox sub-directory (like ~/fj/ff-dev/.mozilla/firefox/abc123.dev), and added a single-profile profiles.ini into the same dir, containing just the entry for the single profile:

[General]
StartWithLastProfile=1

[Profile0]
Name=dev
IsRelative=1
Path=abc123.dev

Then I adjusted my startup script for each profile to use Firejail with the separate home dir:

#!/bin/sh
firejail --private=~/fj/ff-dev firefox

Minimum Filesystem Access

I've also been experimenting with custom Firejail profiles to give Firefox just the minimum access to the filesystem it needs to work. I've found that the following profile (saved as ~/.config/firejail/firefox.profile) on Ubuntu 15.04 enables Firefox to use the system's fonts (private-etc fonts), timezone settings (private-etc localtime), and DNS (private-etc resolve.conf and noblacklist /run/resolveconf); my custom DNS overrides from etc/hosts (private-etc hosts,nsswitch.conf); and DRM flash videos from Hulu, Amazon, etc (private-etc alternatives and noblacklist /run/dbus,/var/cache/hald):

noblacklist /run/dbus
noblacklist /run/resolvconf
noblacklist /run/user
noblacklist /var/cache/hald
noblacklist /var/run
blacklist /boot
blacklist /cdrom
blacklist /lost+found
blacklist /media
blacklist /mnt
blacklist /opt
blacklist /proc
blacklist /run/*
blacklist /sbin
blacklist /srv
blacklist /sys
blacklist /usr/sbin
blacklist /var/*
private-dev
private-etc alternatives,firefox,fonts,hosts,localtime,nsswitch.conf,resolv.conf
read-only /bin
read-only /lib
read-only /lib64
read-only /usr
tmpfs /tmp
caps.drop all
seccomp
netfilter
noroot

I've also built a similar custom Firejail profile for Chrome (saved as ~/.config/firejail/google-chrome.profile) which allows for the same (needing a little less access to run its own built-in version of flash):

noblacklist /opt/google
noblacklist /run/resolvconf
noblacklist /run/user
blacklist /boot
blacklist /cdrom
blacklist /lost+found
blacklist /media
blacklist /mnt
blacklist /opt/*
blacklist /proc
blacklist /run/*
blacklist /sbin
blacklist /srv
blacklist /sys
blacklist /usr/sbin
blacklist /var
private-dev
private-etc alternatives,chromium-browser,fonts,hosts,localtime,nsswitch.conf,resolv.conf
read-only /bin
read-only /lib
read-only /lib64
read-only /usr
tmpfs /tmp
caps.drop all
seccomp
netfilter
noroot

Sunday, April 15, 2012

Fastest Ext4 Options

When I got a new external hard drive for storing mp3s, recorded TV shows, etc, I wanted to know how to set it up most optimally (using the ext4 filesystem). I ended up following Luca Spiller's Ext4 Options for a Media Drive for the most part, but with a few tweaks:

Formatting the Drive

After plugging in the hard drive and running sudo fdisk -l to check what name the OS had assigned it (/dev/sdb1), I formatted the drive with the following options: -m 0 to create no extra room for root (I don't intend to use it as a boot drive); and -L bb to assign it a label of bb (so I can reference it by label in my /etc/fstab):

sudo mkfs.ext4 -m 0 -L bb /dev/sdb1
Configuring the Drive Options

Then I updated my /etc/fstab configuration with an entry for the new drive. Beyond the filesystem permission options of user, rw, exec, and suid (the order of which is significant), and the noauto option (to ignore the drive when booting), I added some options that make writing data less safe — but faster. Use man mount to get a brief description of these and all other possible options:

LABEL=bb /mnt/bb ext4 user,rw,exec,suid,noauto,noatime,nobh,nobarrier,commit=60,data=writeback,journal_async_commit 0 0

If this was an internal hard drive, or one that I intended to have connected at all times, I would have skipped the permissions and noauto options (so as to use the default permissions and allow it to auto-mount at boot time), and would have just specified the performance options:

LABEL=bb /mnt/bb ext4 noatime,nobh,nobarrier,commit=60,data=writeback,journal_async_commit 0 2
Creating the Drive Mount Point

In /etc/fstab I had configured the drive's mount point as /mnt/bb, so I created it and set its owner to myself (so I could mount the drive as a regular user, since I had included the user option in the /etc/fstab config):

sudo mkdir /mnt/bb && sudo chown justin:justin /mnt/bb
Mounting the Drive

Now whenever I plug in the drive, I can mount it as a regular user; either via its label:

mount -L bb

Or via its mount point:

mount /mnt/bb