Initial commit of mount.efs

Includes a mount helper utility (mount.efs), an associated watchdog
process, unit tests, and build scripts.

The utility simplifies mounting and using EFS file systems and
optionally enables clients to mount over a TLS tunnel.

.github/PULL_REQUEST_TEMPLATE.md

+*Issue #, if available:*
+
+*Description of changes:*
+
+
+By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

.gitignore

+amazon-efs-utils*
+!dist/amazon-efs-utils.control
+!dist/amazon-efs-utils.spec
+
+.coverage
+.pytest_cache
+
+*.pyc
+
+__pycache__/
+build/
+

CONTRIBUTING.md

+# Contributing Guidelines
+
+Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 
+documentation, we greatly value feedback and contributions from our community.
+
+Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 
+information to effectively respond to your bug report or contribution.
+
+
+## Reporting Bugs/Feature Requests
+
+We welcome you to use the GitHub issue tracker to report bugs or suggest features.
+
+When filing an issue, please check [existing open](https://github.com/aws/efs-utils/issues), or [recently closed](https://github.com/aws/efs-utils/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already 
+reported the issue. Please try to include as much information as you can. Details like these are incredibly useful:
+
+* A reproducible test case or series of steps
+* The version of our code being used
+* Any modifications you've made relevant to the bug
+* Anything unusual about your environment or deployment
+
+
+## Contributing via Pull Requests
+Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that:
+
+1. You are working against the latest source on the *master* branch.
+2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already.
+3. You open an issue to discuss any significant work - we would hate for your time to be wasted.
+
+To send us a pull request, please:
+
+1. Fork the repository.
+2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change.
+3. Ensure local tests pass.
+4. Commit to your fork using clear commit messages.
+5. Send us a pull request, answering any default questions in the pull request interface.
+6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation.
+
+GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 
+[creating a pull request](https://help.github.com/articles/creating-a-pull-request/).
+
+
+## Finding contributions to work on
+Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels ((enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/aws/efs-utils/labels/help%20wanted) issues is a great place to start. 
+
+
+## Code of Conduct
+This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 
+For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 
+opensource-codeofconduct@amazon.com with any additional questions or comments.
+
+
+## Licensing
+
+See the [LICENSE](https://github.com/aws/efs-utils/blob/master/LICENSE) file for our project's licensing. We will ask you confirm the licensing of your contribution.
+
+We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes.

LICENSE

+MIT License
+
+Copyright 2017 Amazon.com, Inc. or its affiliates. 
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

Makefile

+#
+# Copyright 2017-2018 Amazon.com, Inc. and its affiliates. All Rights Reserved.
+#
+# Licensed under the MIT License. See the LICENSE accompanying this file
+# for the specific language governing permissions and limitations under
+# the License.
+#
+
+PACKAGE_NAME = amazon-efs-utils
+SOURCE_TARBALL = $(PACKAGE_NAME).tar.gz
+SPECFILE = $(PACKAGE_NAME).spec
+BUILD_DIR = build/rpmbuild
+export PYTHONPATH := $(shell pwd)/src
+
+clean:
+	rm -rf $(BUILD_DIR)
+	rm -f $(SOURCE_TARBALL)
+	rm -f $(SPECFILE)
+
+tarball: clean
+	mkdir -p $(PACKAGE_NAME)
+	cp -rp dist $(PACKAGE_NAME)
+	cp -rp src $(PACKAGE_NAME)
+	tar -czf $(SOURCE_TARBALL) $(PACKAGE_NAME)/*
+
+specfile: clean
+	ln -sf dist/$(SPECFILE) $(SPECFILE)
+
+sources: tarball specfile
+
+rpm-only:
+	mkdir -p $(BUILD_DIR)/{SPECS,COORD_SOURCES,DATA_SOURCES,BUILD,RPMS,SOURCES,SRPMS}
+	cp $(SPECFILE) $(BUILD_DIR)/SPECS
+	cp $(SOURCE_TARBALL) $(BUILD_DIR)/SOURCES
+	rpmbuild -ba --define "_topdir `pwd`/$(BUILD_DIR)" $(BUILD_DIR)/SPECS/$(SPECFILE)
+	cp $(BUILD_DIR)/RPMS/*/*rpm build
+
+rpm: sources rpm-only
+
+.PHONY: test
+test:
+	pytest
+	flake8
\ No newline at end of file

NOTICE

+ efs-utils
+ Copyright 2017-2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.

README.md

+# efs-utils
+
+Utilities for Amazon Elastic File System (EFS)
+
+The `efs-utils` package has been verified against the following Linux distributions:
+
+| Distribution | Package Type | `init` System |
+| ------------ | ------------ | ------------- |
+| Amazon Linux 2017.09 | `rpm` | `upstart` |
+| Amazon Linux 2 | `rpm` | `systemd` |
+| CentOS 7 | `rpm` | `systemd` |
+| RHEL 7 | `rpm`| `systemd` |
+| Debian 9 | `deb` | `systemd` |
+| Ubuntu 16.04 | `deb` | `systemd` |
+
+## Installation
+
+### On Amazon Linux distributions
+
+For those using Amazon Linux or Amazon Linux 2, the easiest way to install `efs-utils` is from Amazon's repositories:
+
+```
+$ sudo yum -y install amazon-efs-utils
+```
+
+### On other Linux distributions
+
+Other distributions require building the package from source and installing it.
+
+- Clone this repository:
+
+```
+$ git clone https://github.com/aws/efs-utils
+$ cd efs-utils
+```
+
+- To build and install an RPM:
+
+```
+$ sudo yum -y install rpm-build
+$ make rpm
+$ sudo yum -y install build/amazon-efs-utils*rpm
+```
+
+- To build and install a Debian package:
+
+```
+$ sudo apt-get update
+$ sudo apt-get -y install binutils
+$ ./build-deb.sh
+$ sudo apt-get -y install ./build/amazon-efs-utils*deb
+```
+
+#### Run tests
+
+- [Set up a virtualenv](http://libzx.so/main/learning/2016/03/13/best-practice-for-virtualenv-and-git-repos.html) for efs-utils
+
+```
+$ virtualenv ~/.envs/efs-utils
+$ source ~/.envs/efs-utils/bin/activate
+$ pip install -r requirements.txt
+```
+
+- Run tests
+
+```
+$ make test
+```
+
+## Usage
+
+### mount.efs
+
+`efs-utils` includes a mount helper utility to simplify mounting and using EFS file systems.
+
+To mount with the recommended default options, simply run:
+
+```
+$ sudo mount -t efs file-system-id efs-mount-point/
+```
+
+To mount automatically with recommended options, add an `/etc/fstab` entry like:
+
+```
+file-system-id efs-mount-point efs _netdev 0 0
+```
+
+To mount over TLS, simply add the `tls` option:
+
+```
+$ sudo mount -t efs -o tls file-system-id efs-mount-point/
+```
+
+To mount over TLS automatically, add an `/etc/fstab` entry like:
+
+```
+file-system-id efs-mount-point efs _netdev,tls 0 0
+```
+
+For more information on mounting with the mount helper, see the [documentation](https://docs.aws.amazon.com/efs/latest/ug/using-amazon-efs-utils.html).
+
+#### amazon-efs-mount-watchdog
+
+`efs-utils` contains a watchdog process to monitor the health of TLS mounts. This process is managed by either `upstart` or `systemd` depending on your Linux distribution, and is started automatically the first time an EFS file system is mounted over TLS.
+
+## License Summary
+
+This code is made available under the MIT license.
+

build-deb.sh

+#!/usr/bin/env sh
+#
+# Copyright 2017-2018 Amazon.com, Inc. and its affiliates. All Rights Reserved.
+#
+# Licensed under the MIT License. See the LICENSE accompanying this file
+# for the specific language governing permissions and limitations under
+# the License.
+#
+
+set -ex
+
+BASE_DIR=$(pwd)
+BUILD_ROOT=${BASE_DIR}/build/debbuild
+
+echo 'Cleaning deb build workspace'
+rm -rf ${BUILD_ROOT}
+mkdir -p ${BUILD_ROOT}
+
+echo 'Creating application directories'
+mkdir -p ${BUILD_ROOT}/etc/amazon/efs
+mkdir -p ${BUILD_ROOT}/etc/init/
+mkdir -p ${BUILD_ROOT}/etc/systemd/system
+mkdir -p ${BUILD_ROOT}/sbin
+mkdir -p ${BUILD_ROOT}/usr/bin
+mkdir -p ${BUILD_ROOT}/var/log/amazon/efs
+
+echo 'Copying application files'
+install -p -m 644 dist/amazon-efs-mount-watchdog.conf ${BUILD_ROOT}/etc/init
+install -p -m 644 dist/amazon-efs-mount-watchdog.service ${BUILD_ROOT}/etc/systemd/system
+install -p -m 644 dist/efs-utils.conf ${BUILD_ROOT}/etc/amazon/efs
+install -p -m 755 src/mount_efs/__init__.py ${BUILD_ROOT}/sbin/mount.efs
+install -p -m 755 src/watchdog/__init__.py ${BUILD_ROOT}/usr/bin/amazon-efs-mount-watchdog
+
+echo 'Copying install scripts'
+install -p -m 755 dist/scriptlets/before-upgrade ${BUILD_ROOT}/preinst
+install -p -m 755 dist/scriptlets/after-install-upgrade ${BUILD_ROOT}/postinst
+install -p -m 755 dist/scriptlets/before-remove ${BUILD_ROOT}/prerm
+install -p -m 755 dist/scriptlets/after-remove ${BUILD_ROOT}/postrm
+
+echo 'Copying control file'
+install -p -m 644 dist/amazon-efs-utils.control ${BUILD_ROOT}/control
+
+echo 'Creating deb binary file'
+echo '2.0'> ${BUILD_ROOT}/debian-binary
+
+echo 'Setting permissions'
+find ${BUILD_ROOT} -type d | xargs chmod 755;
+
+echo 'Creating tar'
+cd ${BUILD_ROOT}
+tar czf control.tar.gz control preinst postinst prerm postrm --owner=0 --group=0
+tar czf data.tar.gz etc sbin usr var --owner=0 --group=0
+cd ${BASE_DIR}
+
+echo 'Building deb'
+DEB=${BUILD_ROOT}/amazon-efs-utils-1.0-1.deb
+ar r ${DEB} ${BUILD_ROOT}/debian-binary
+ar r ${DEB} ${BUILD_ROOT}/control.tar.gz
+ar r ${DEB} ${BUILD_ROOT}/data.tar.gz
+
+echo 'Copying deb to output directory'
+cp ${BUILD_ROOT}/amazon-efs-utils*deb build/

dist/amazon-efs-mount-watchdog.conf

+#
+# Copyright 2017-2018 Amazon.com, Inc. and its affiliates. All Rights Reserved.
+#
+# Licensed under the MIT License. See the LICENSE accompanying this file
+# for the specific language governing permissions and limitations under
+# the License.
+#
+
+description     "Amazon EFS Mount Watchdog"
+author          "Amazon.com"
+
+# Uncomment these lines to start amazon-efs-mount-watchdog automatically on boot
+# start on (runlevel [345] and started network)
+# stop on (runlevel [!345] or stopping network)
+
+respawn
+respawn limit 0 15
+
+exec /usr/bin/env amazon-efs-mount-watchdog

dist/amazon-efs-mount-watchdog.service

+#
+# Copyright 2017-2018 Amazon.com, Inc. and its affiliates. All Rights Reserved.
+#
+# Licensed under the MIT License. See the LICENSE accompanying this file
+# for the specific language governing permissions and limitations under
+# the License.
+#
+
+[Unit]
+Description=amazon-efs-mount-watchdog
+
+[Service]
+Type=simple
+ExecStart=/usr/bin/env amazon-efs-mount-watchdog
+KillMode=process
+Restart=on-failure
+RestartSec=15s
+
+[Install]
+WantedBy=multi-user.target

dist/amazon-efs-utils.control

+Package: amazon-efs-utils
+Architecture: all
+Version: 1.0
+Section: utils
+Depends: python|python2, nfs-common, stunnel4 (>= 4.56)
+Priority: optional
+Copyright: MIT License
+Maintainer: Amazon.com, Inc. <efs-utils@amazon.com>
+Description: This package provides utilities for simplifying the use of EFS file systems

dist/amazon-efs-utils.spec

+#
+# Copyright 2017-2018 Amazon.com, Inc. and its affiliates. All Rights Reserved.
+#
+# Licensed under the MIT License. See the LICENSE accompanying this file
+# for the specific language governing permissions and limitations under
+# the License.
+#
+
+%if 0%{?amzn1}
+%global python_requires system-python
+%global with_systemd 0
+%else
+%global python_requires python2
+%global with_systemd 1
+%endif
+
+Name      : amazon-efs-utils
+Version   : 1.1
+Release   : 1%{?dist}
+Summary   : This package provides utilities for simplifying the use of EFS file systems
+
+Group     : Amazon/Tools
+License   : MIT
+URL       : https://aws.amazon.com/efs
+
+Packager  : Amazon.com, Inc. <http://aws.amazon.com>
+Vendor    : Amazon.com
+
+BuildArch : noarch
+
+Requires  : nfs-utils
+Requires  : stunnel >= 4.56
+Requires  : %{python_requires}
+
+%if %{with_systemd}
+BuildRequires    : systemd
+%{?systemd_requires}
+%else
+Requires(post)   : /sbin/chkconfig
+Requires(preun)  : /sbin/service /sbin/chkconfig
+Requires(postun) : /sbin/service
+%endif
+
+Source    : %{name}.tar.gz
+
+%description
+This package provides utilities for simplifying the use of EFS file systems
+
+%prep
+%setup -n %{name}
+
+%install
+mkdir -p %{buildroot}%{_sysconfdir}/amazon/efs
+%if %{with_systemd}
+mkdir -p %{buildroot}%{_unitdir}
+install -p -m 644 %{_builddir}/%{name}/dist/amazon-efs-mount-watchdog.service %{buildroot}%{_unitdir}
+%else
+mkdir -p %{buildroot}%{_sysconfdir}/init
+install -p -m 644 %{_builddir}/%{name}/dist/amazon-efs-mount-watchdog.conf %{buildroot}%{_sysconfdir}/init
+%endif
+
+mkdir -p %{buildroot}/sbin
+mkdir -p %{buildroot}%{_bindir}
+mkdir -p %{buildroot}%{_localstatedir}/log/amazon/efs
+
+install -p -m 644 %{_builddir}/%{name}/dist/efs-utils.conf %{buildroot}%{_sysconfdir}/amazon/efs
+install -p -m 755 %{_builddir}/%{name}/src/mount_efs/__init__.py %{buildroot}/sbin/mount.efs
+install -p -m 755 %{_builddir}/%{name}/src/watchdog/__init__.py %{buildroot}%{_bindir}/amazon-efs-mount-watchdog
+
+%files
+%defattr(-,root,root,-)
+%if %{with_systemd}
+%{_unitdir}/amazon-efs-mount-watchdog.service
+%else
+%config(noreplace) %{_sysconfdir}/init/amazon-efs-mount-watchdog.conf
+%endif
+/sbin/mount.efs
+%{_bindir}/amazon-efs-mount-watchdog
+/var/log/amazon
+
+%config(noreplace) %{_sysconfdir}/amazon/efs/efs-utils.conf
+
+%if %{with_systemd}
+%post
+%systemd_post amazon-efs-mount-watchdog.service
+
+%preun
+%systemd_preun amazon-efs-mount-watchdog.service
+
+%postun
+%systemd_postun_with_restart amazon-efs-mount-watchdog.service
+
+%else
+
+%preun
+if [ $1 -eq 0 ]; then
+   /sbin/stop amazon-efs-mount-watchdog &> /dev/null || true
+fi
+
+%postun
+if [ $1 -eq 1 ]; then
+    /sbin/restart amazon-efs-mount-watchdog &> /dev/null || true
+fi
+
+%endif
+
+%clean

dist/efs-utils.conf

+#
+# Copyright 2017-2018 Amazon.com, Inc. and its affiliates. All Rights Reserved.
+#
+# Licensed under the MIT License. See the LICENSE accompanying this file
+# for the specific language governing permissions and limitations under
+# the License.
+#
+
+[DEFAULT]
+logging_level = INFO
+logging_max_bytes = 1048576
+logging_file_count = 10
+
+[mount]
+dns_name_format = {fs_id}.efs.{region}.amazonaws.com
+stunnel_debug_enabled = false
+
+# Validate the certificate hostname on mount. Requires stunnel >= 5.15.
+stunnel_check_cert_hostname = false
+
+# Use OCSP to check certificate validity. Requires stunnel >= 5.10.
+stunnel_check_cert_validity = false
+
+# Define the port range that the TLS tunnel will choose from
+port_range_lower_bound = 20049
+port_range_upper_bound = 20449
+
+[mount-watchdog]
+enabled = true
+poll_interval_sec = 1
+unmount_grace_period_sec = 30

dist/scriptlets/after-install-upgrade

+if [ $(cat /proc/1/comm) = init ]; then
+    /sbin/restart amazon-efs-mount-watchdog &> /dev/null || true
+elif [ $(cat /proc/1/comm) = systemd ]; then
+    systemctl try-restart amazon-efs-mount-watchdog.service &> /dev/null || true
+fi

dist/scriptlets/after-remove

+if [ $(cat /proc/1/comm) = systemd ]; then
+    systemctl daemon-reload
+fi

dist/scriptlets/before-remove

+if [ $(cat /proc/1/comm) = init ]; then
+    /sbin/stop amazon-efs-mount-watchdog &> /dev/null || true
+elif [ $(cat /proc/1/comm) = systemd ]; then
+    systemctl --no-reload disable amazon-efs-mount-watchdog.service &> /dev/null || true
+    systemctl stop amazon-efs-mount-watchdog.service &> /dev/null || true
+fi

dist/scriptlets/before-upgrade

+if [ $(cat /proc/1/comm) = init ]; then
+    /sbin/stop amazon-efs-mount-watchdog &> /dev/null || true
+elif [ $(cat /proc/1/comm) = systemd ]; then
+    systemctl stop amazon-efs-mount-watchdog.service &> /dev/null || true
+fi

requirements.txt

+attrs==17.4.0
+configparser==3.5.0
+coverage==4.5
+enum34==1.1.6
+flake8==3.5.0
+funcsigs==1.0.2
+mccabe==0.6.1
+mock==2.0.0
+pbr==3.1.1
+pluggy==0.6.0
+py==1.5.2
+pycodestyle==2.3.1
+pyflakes==1.6.0
+pytest==3.4.0
+pytest-cov==2.5.1
+pytest-html==1.16.1
+pytest-metadata==1.6.0
+pytest-mock==1.6.3
+six==1.11.0

setup.cfg

+[flake8]
+max-line-length = 130
+exclude = test
+
+[tool:pytest]
+addopts =
+    --verbose
+    --html build/pytest/index.html
+    --cov mount_efs
+    --cov watchdog
+    --cov-report html:build/coverage
+    --cov-fail-under 80

src/mount_efs/__init__.py

+#!/usr/bin/env python
+#
+# Copyright 2017-2018 Amazon.com, Inc. and its affiliates. All Rights Reserved.
+#
+# Licensed under the MIT License. See the LICENSE accompanying this file
+# for the specific language governing permissions and limitations under
+# the License.
+#
+#
+# Copy this script to /sbin/mount.efs and make sure it is executable.
+#
+# You will be able to mount an EFS file system by its short name, by adding it
+# to /etc/fstab. The syntax of an fstab entry is:
+#
+# [Device] [Mount Point] [File System Type] [Options] [Dump] [Pass]
+#
+# Add an entry like this:
+#
+#   fs-deadbeef     /mount_point    efs     _netdev         0   0
+#
+# Using the 'efs' type will cause '/sbin/mount.efs' to be called by 'mount -a'
+# for this file system. The '_netdev' option tells the init system that the
+# 'efs' type is a networked file system type. This has been tested with systemd
+# (Amazon Linux 2, CentOS 7, RHEL 7, Debian 9, and Ubuntu 16.04), and upstart
+# (Amazon Linux 2017.09).
+#
+# Once there is an entry in fstab, the file system can be mounted with:
+#
+#   sudo mount /mount_point
+#
+# The script will add recommended mount options, if not provided in fstab.
+
+import getpass
+import json
+import logging
+import os
+import random
+import re
+import socket
+import subprocess
+import sys
+import threading
+
+from contextlib import contextmanager
+from logging.handlers import RotatingFileHandler
+
+try:
+    import ConfigParser
+except ImportError:
+    from configparser import ConfigParser
+
+try:
+    from urllib2 import urlopen, URLError
+except ImportError:
+    from urllib.error import URLError
+    from urllib.request import urlopen
+
+VERSION = 1.0
+
+CONFIG_FILE = '/etc/amazon/efs/efs-utils.conf'
+CONFIG_SECTION = 'mount'
+
+LOG_DIR = '/var/log/amazon/efs'
+LOG_FILE = 'mount.log'
+
+STATE_FILE_DIR = '/var/run/efs'