Send Mail from your Raspberry Pi

I have to go searching for how to do this every time. This time I thought I would memorialize the instructions. I found this in an article written by Stacy Prowell on Medium.com.

The first thing you need is some additional software. I am going to use Postfix, which is a mail transfer agent (MTA) — that is, something that knows how to talk to other MTAs to send and receive email. In particular, Postfix supports the simple mail transfer protocol (SMTP), so it can talk to nearly any other MTA out there, including Google’s Gmail. In case you don’t like Gmail, there are guides online like this one for getting Postfix to talk to other SMTP servers.

1. Install Postfix

Run the following command at the prompt to install Postfix along with simple authentication layer security (SASL), which Postfix will use to connect to Gmail.

sudo apt install postfix libsasl2-modules

During the installation, you will be asked about how the mail server should operate. You want an Internet Site, where email is sent and received directly using SMTP, so select that option.

Screenshot of package configuration screen
So many choices

Now you’ll be asked for the “system mail name.” You should use your hostname (raspberrypi, for instance) or, if you have a fully-qualified domain name for your network via your ISP or a service like DYN, then you can use that. Don’t stress over this; you can modify it later by editing the /etc/postfix/main.cf file if you need to.

Screenshot of a package configuration dialog
Set the server name

2. Get an Application Password for Postfix from Google

have Google set up for two-factor authentication (2FA), so how will the Pi be able to send an email? Well, it turns out you can get Google to generate an application password, which is a password to allow a specific application to connect.

To get an application password head to: https://myaccount.google.com and log in.

Screenshot of Google account options
Account security settings

Select “Security” from the list on the left.

Screenshot of Google account security options
Manage your application passwords

Note that I have two-step verification turned on. You might or might not; either way, creating an application password for each application is a good idea. If you lose a device or it is stolen, you can revoke the application password and not have to change your existing password or passwords for other applications.

There should be a box for “Signing in to Google” that contains an option for “App passwords.” Click on that. You might have to sign in again at this point (I did). This should take you to the page to manage application passwords. The example account I am using doesn’t have any.

Screenshot of app password manager
Create a new application password

Now click on “Select app” and select Mail. Then click on “Select device” and select Other. You’ll need to enter the name of the device (for this example it is raspberrypi4). Click Generate to create the application password.

Screenshot of new application password
A new application password! Don’t get excited; I’ve already deleted it.

You should now be presented with a new application password. This is the text in the yellow block in the image above. Success! Do not click done! Copy the app password first, or write it down. You’ll need it and you can’t display it again.

3. Configure SASL

(A lot of the content of this section is taken from the Postfix SASL HowTo.)

We are almost done. The next thing to do is to add the app password you just generated to the SASL configuration. Run the following command.

sudo nano -B /etc/postfix/sasl/sasl_passwd

This command will open the file in an editor. It is likely the file does not exist, and you will see an empty file. Add the following line, replacing username and password with your Gmail username and the application password you just generated (don’t include the spaces).

[smtp.gmail.com]:587 username@gmail.com:password

This line tells SASL that when it connects to the host smtp.gmail.com at port 587 to download mail, it should use the given username and password to connect. Exit and save with CTRL+x, y, and Enter.

This file contains a “clear text” password. Run the following command to protect that file.

sudo chmod u=rw,go= /etc/postfix/sasl/sasl_passwd

This command sets the user permissions (root) to read and write and removes any permissions for the group and others. (Fans of the numerical form of chmod will recognize this as 0600.)

Now turn this file into a hash file for Postfix. Run the following command.

sudo postmap /etc/postfix/sasl/sasl_passwd

This command will create a new file named sasl_passwd.db in the same directory. It should already have the permissions set correctly, but just in case, let’s also explicitly set the permissions.

sudo chmod u=rw,go= /etc/postfix/sasl/sasl_passwd.db

4. Configure Postfix

Now let’s finish up the configuration of Postfix. Run the following commands.

sudo cp /etc/postfix/main.cf !#$.dist
sudo nano /etc/postfix/main.cf

Find the line (near the bottom) that starts with relayhost =. Here is where we specify that we want to use Google’s SMTP server as our relay host, and this must match what we put in /etc/postfix/sasl/sasl_passwd. Change the line so it looks as follows.

relayhost = [smtp.gmail.com]:587

Next, add the following to the end of the file. (Documentation for these options and others can be found here.)

# Enable authentication using SASL.
smtp_sasl_auth_enable = yes
# Use transport layer security (TLS) encryption.
smtp_tls_security_level = encrypt
# Do not allow anonymous authentication.
smtp_sasl_security_options = noanonymous
# Specify where to find the login information.
smtp_sasl_password_maps = hash:/etc/postfix/sasl/sasl_passwd
# Where to find the certificate authority (CA) certificates.
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt

Save and exit with CTRL+x, y, and Enter, and then restart Postfix with the following command.

sudo systemctl restart postfix

5. Testing Email

Next, let’s verify that we can send an email via Google’s SMTP server. Type the following at the command line, replacing username with your Gmail user name (so the email goes to you).

sendmail username@gmail.com
Subject: It works!
Hey, it works!
.

The first line uses the sendmail command to send an email to the specified recipients (you can list more than one). The subsequent lines optionally specify mail headers like the subject, and then the body of the email. The entire thing is terminated by a period. Note that this is just one way to send email using the sendmail command. Later we will use sendmail to send the output of commands.

Check your email. If you got an email from yourself (remember: Postfix is using your credentials) then everything is working. If you didn’t, then you should check a few places on the system.

Checking for Problems

The most obvious problems are an inability to reach the Gmail server and a failure to authenticate with the Gmail server.

Inability to reach the Gmail SMTP servers

Make sure the network is connected and see if you can reach the Gmail SMTP server. The easiest way to do this is to “ping” smtp.gmail.com. If you see a failure message like “ping: smtp.gmail.com: Name or service not known,” then you should check your network connectivity. (It is possible that your ISP or the Google service itself is down… but this is less likely.)

Failure to authenticate with the Gmail SMTP servers

To see this, check the file /var/log/syslog for messages from postfix. If you see a message like the one below, your credentials were not accepted.

Dec 10 08:04:01 raspberrypi postfix/smtp[18329]: 331F960582: SASL authentication failed; server smtp.gmail.com[108.177.122.108] said: 535-5.7.8 Username and Password not accepted. Learn more at?535 5.7.8  https://support.google.com/mail/?p=BadCredentials c188sm1372198ywb.56 - gsmtp

Check that the content of /etc/postfix/sasl/sasl_passwd is exactly what it should be, and make sure you ran the postmap command to create the hash file. If everything looks correct, or if you don’t have your application password written down, go back to Google account management and delete the old application password, create a new one, and carefully add it to /etc/postfix/sasl/sasl_passwd and then run the postmap command given earlier.

Adding Email Aliases

You can add email aliases by editing the file /etc/aliases and then running the newaliases command. This can be used to tell Postfix how to handle local email addresses.

For example, perhaps you want email to the pi user to go to your Gmail account. You would run the following command to edit the aliases file.

sudo nano -R /etc/aliases

Next, you would add the following line, where username is your Gmail username.

pi: username@gmail.com

Finally, tell the system about the new mail aliases by running the following command.

sudo newaliases

Now mail sent to pi will be forwarded along by Postfix to your Gmail account.

Depending on what you are using the Pi for, you might forward postmaster, webmaster, or other names. Note that these don’t have to correspond to any local account.

Sending Text Messages

A computer in our server room sends me a text message each morning using a dedicated cellular long term evolution (LTE) modem. The Pi doesn’t have one (though you can buy them; for instance, here’s one), but if you’ve successfully followed the above instructions, your Pi can send email, and it turns out you can send a text message by first sending an email.

First, a word of warning. Sending an email over a connection to Gmail isn’t likely to cost you any more than you are already paying for internet connectivity. But receiving a text message on your phone just might. Check your plans, and be courteous of other people who might not want unsolicited text messages from you.

Email to Text

Cellular providers will convert emails sent to a special address into text messages, and forward them along. Text replies may be converted back to a return email, but this is less certain. The two articles below cover this in some detail and provide information for both short message service (SMS) and multimedia message service (MMS), where supported.

Here are the top four North American providers and the information for each. (Apologies to non-North American readers; there are far too many wireless services around the world for me to try to include them all.)

Service Name SMS Email Suffix MMS Email Suffix
AT&T Mobility @txt.att.net @mms.att.net
Verizon Wireless @vtext.com @vzwpix.com
T-Mobile @tmomail.net @tmomail.net (same)
Sprint Corporation @messaging.sprintpcs.com @pm.sprint.com
view raw email-to-text.csv hosted with ❤ by GitHub

You use these by finding the provider for the recipient of the message, and then sending an email to the ten-digit (North American) phone number followed by the appropriate email suffix.

For instance, suppose your cell phone number is (724) 555–1212, and your provider is Verizon. Then you can send yourself a text by sending an email message to the address 7245551212@vtext.com. These SMS messages are typically limited to 140 characters.

You can also send MMS messages, but this is a bit more complex.

Email to Text from the Raspberry Pi

My carrier is Sprint, and my number is (not really) 724–555–1212. So to send a text to my phone I can send a short text email to 7245551212@messaging.sprintpcs.com. Let’s try a test.

At the prompt of the Raspberry Pi, try the following (replacing the address with whatever is correct for your cell phone).

echo "Test" | sendmail 7245551212@messaging.sprintpcs.com

You should receive a text on your phone.

Screenshot of a text message
Success!

Great! If you don’t, check your Gmail for a delivery failure notice, and make sure you are using the correct address for the recipient (you, in this case).

Reply to Text

What happens if you reply to a text? Well, that depends on your service provider. For my provider (Sprint) an email is created and sent.

Reply email
Talking back to the text

You can reply to this email, and the result will be a (possibly very ugly) text message. Communication achieved!

Note that if you are getting your hopes up on automatically processing the replies, you should be careful. The reply above was sent as a base 64 encoded rich text file. That is, you may have to do some work if you want true two-way communication.

Things to Say

Now that you can send email and text messages from the Pi, what should the Pi say? Well, as mentioned at the start, I set my Pi up to be a file server, and so one thing I would like is a report on the file system usage each morning when I get up. I’m also mildly paranoid, so I’d like to know when someone logs into the Pi. One is almost trivial. The other requires a bit more work.

This would be a good time to add an alias for text messages. For example, you might add the following line to your /etc/aliases and then run sudo newaliases as described previously.

alert: 7245551212@messaging.sprintpcs.com

From now on you can just send email to alert, and Postfix will turn it into a text message to your phone.

Reporting File System Usage

I am using two drives to store information: /data/a and /data/b (I’m not that creative). I can check on the file system used with the following command.

df --output=target,pcent /data/a /data/b

(I only want to know about those two file systems; if I did not include them, the df command would list all the mounted file systems… which might be what you want in some cases.)

This tells me exactly what I want to know: how much space is used in each file system. If this space gets too high, I know I need to add more storage. This is the output from the above command as I write this.

Mounted on Use%
/data/a 3%
/data/b 2%

It’s short and simple, under 140 characters, and a perfect thing to text to me in the morning to let me know the status of my file server. I can drop the header line and send the output with the following command, using the alert alias we created at the start of this section.

df --output=target,pcent /data/a /data/b | \
tail +2 | /usr/sbin/sendmail alert
A screenshot of text messages
Success! Information you can ignore until it gets out of hand.

(The tail +2 tells the system to copy all output from the prior command, starting at the second line… so this skips the header line.)

Great! Now I want this to run as a command every morning at 6:00 am. For that, I will use the Linux cron utility.

Every user has a special file called the crontab that contains information on commands to execute, when, how often, etc. Do not edit this file directly! There is a special command, crontab, for examining and editing this file.

The following command will list the content of the user’s crontab file.

crontab -l

By default, each user gets a crontab file with a helpful usage message as a series of comments. You should leave these comments in for reference!

# Edit this file to introduce tasks to be run by cron.               
#
# Each task to run has to be defined through a single line
# indicating with different fields when the task will be run
# and what command to run for the task
#
# To define the time you can provide concrete values for
# minute (m), hour (h), day of month (dom), month (mon),
# and day of week (dow) or use '*' in these fields (for 'any').
#
# Notice that tasks will be started based on the cron's system
# daemon's notion of time and timezones.
#
# Output of the crontab jobs (including errors) is sent through
# email to the user the crontab file belongs to (unless redirected).
#
# For example, you can run a backup of all your user accounts
# at 5 a.m every week with:
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
#
# For more information see the manual pages of crontab(5) and cron(8)
#
# m h dom mon dow command

Now let’s add a line to generate the file system usage message every morning at 6:00 am. Note that we want to run more than one command, so we need to enclose the whole thing in parentheses. If we were running a single command we would not need the parentheses.

To edit the crontab, run crontab -e. This will open your crontab in an editor. Add the following line, below the format comment, save, and exit.

# m h  dom mon dow   command
0 6 * * * ( df --output=target,pcent /data/a /data/b | tail +2 | /usr/sbin/sendmail alert )

The minutes (m) is zero, the hour (h) is 6 (using 24 hour time), we want every day of the month (dom), every month (mon), and every day of the week (dow), so we set those three to asterisks (*). Alternately, if I just wanted a report every Friday afternoon at 5:30 pm, I would write the following.

# m h  dom mon dow   command
30 17 * * FRI ( df --output=target,pcent /data/a /data/b | tail +2 | /usr/sbin/sendmail alert )

If this seems much too tricky, don’t lose hope. Head over to the Crontab Guru site for help. You can (and I do) have multiple lines in your crontab to do a variety of things. If you want to know more about how crontab works, the file format, etc., see Ranjan Bajracharya’s article on cron jobs.

Reporting SSH Logins

I have the Pi set up to allow remote logins using two-factor authentication. I wrote a short article on how to do that, too.

Setting Up SSH and 2FA on a Raspberry PiLog in from anywhere using SSH and Google Authenticatormedium.com

Because the Pi is accessible remotely, I’d like to be alerted when someone logs in via SSH. Doing that is pretty easy, it turns out. (Of course, this is only going to work is you have enabled SSH.)

First, create a new script. We’ll put this in the /etc folder.

sudo nano /etc/notify-ssh-login.sh

The content of the file should be as follows. (Note that this script depends on the alert alias created at the start of this section. It is a Really Good Idea to try to capture this sort of information in a single place and not scatter it around the system!)

#!/bin/bashPATH=/bin:/usr/bin:/usr/sbin
SUBJ="Alert - Remote SSH access from ${PAM_USER}"if [ -z "$PAM_TYPE" -o "$PAM_TYPE" == "open_session" ]
thensendmail alert <<END
$(hostname) $(date): LOGIN by ${PAM_USER} from ${PAM_RHOST}
$(who)
ENDelsesendmail alert <<END
$(hostname) $(date): LOG OUT by ${PAM_USER} from ${PAM_RHOST}
ENDfi

Next, we need to arrange for the Linux pluggable authentication module (PAM) system to invoke this script whenever someone logs in or logs out via an SSH session. Run the following command.

sudo nano -R /etc/pam.d/sshd

Let’s protect the file against changes.

sudo chmod go-w /etc/notify-ssh-login.sh

Add the following lines at the end of the file.

# Notify on successful login / log out.
session optional pam_exec.so /etc/notify-ssh-login.sh

Now log into the Pi via SSH and make sure you get a text message. You should get a shorter message when you log out.

Screenshot of text messages reporting login and log out
Watching log in and log out. The list of users in the first message includes the console user (tty1)

You can do a lot of other things with PAM; details are beyond this short document, but there are many excellent articles that cover this important system.

Why do you guys insist on using Linux?

There’s been some recent discussions/arguments on forums about the use of Unix variants for these DIY projects. There is no argument that Windows has a crushing lead in the desktop OS market share. Those of us who do this, with some notable (notorious?) exceptions, do this for free. In essence, we do it just for the happiness we get seeing others use our work. If that’s our goal, why not “cater to the masses?” Surely it would increase the adoption rate?

I’m going to use BrewPi Remix as an example since I’m most familiar with it. Unless someone has a better example, I believe it is the oldest surviving DIY/Open Source project, released in 2011. As such, I believe it was also the original project which brought the Raspberry Pi to our projects. While Elco did the original BrewPi work on an old router, the Raspberry Pi released in 2012 and, by February 2015, had sold more than 5 million boards. It was arguably the perfect platform for his use.

I deliver IT projects professionally. When it all comes down to it, my clients are concerned with three things: Cost, schedule, and performance. My experience is that this method of determining success and value applies to these DIY projects as well. I’ll attempt to use the same method to demonstrate the reasons behind these decisions, and hopefully explain why Elco and then the rest of us have chosen this ecosphere.

Cost

Right this moment (January 11th, 2020), the Raspberry Pi Zero W (a wireless variant) is $31.99 on Amazon (US), including a case, SD card, and power supply. That is everything you need if you already have a monitor mouse and keyboard lying around. I dare someone to show me another network-capable computer with the OS license included at that price point.

At this point in the argument, a few people generally point out that they have an old Whizbang 2000 computer with Windows 95 on it that’s perfectly capable of doing the work. This counter-argument is called “confirmation bias.” Applied to our problem, it is exceedingly easy on the Internet today to find someone to disprove any statement. That does not mean that this is true for everyone or even the majority of people. In my experience, it is quite the contrary. The most common thing I hear is, “can you give me a 1 to n list of everything I need?” People are buying this for these projects and startup cost is important.

Even if a person has an old computer, that old computer has value. Otherwise, they would not have saved it in the first place. That value necessarily needs to be applied to the equation. If it is a “perfectly good computer,” it’s likely worth more than $31. If I completely ignore that argument, I am still left with a “cost” to prepare that old computer for use. That cost is my (or my users’) time.

I’ve not stumbled across an “old but useful” computer that was not set aside for a good reason. Maybe it is slow, maybe it is out of space, but before we use it, it needs a little work. Uninstall old programs, update the antivirus, apply patches, maybe run the system recovery to revert it to factory state. That takes a few hours of one’s time. There is always a balancing act for people to determine how much value their time has. This balance is certainly a personal decision, but one that the individual should consider carefully.

Licensing is a cost. Granted, most people forget about this when they buy their PC’s, but those licenses have value. A retail copy of Windows 10 Home edition is currently about $130. A copy of Raspbian is free as in free beer. There is no cost for Raspbian.

There is also a cost for the power consumed by a computer. I do not know many people who consider this, but the impact is not insignificant. According to one person who took the time to measure, 100 watts is a good average to use for cost estimation.   

If you take an average of all 50 states, 13.31 cents per kilowatt-hour (kWh) is a good number to use. That 100-watt average for the PC equals 2.4 kWh per day or 876 kWh for a year. Running that PC costs $116.60 for the year. The Raspberry Pi Zero power supply I listed above uses a maximum (the PC was a measured average) of 2.5 Amps of 5V power, which is 12.5 watts. If that humble Pi (see what I did there?) were running at full tilt for a year, it would cost $14.57 per year. The Pi comes in saving you $102.03 per year. If you only count a single year and you count your “perfectly good PC” as entirely free, you still save $70.04 a year after buying a brand new Pi.

If you only count expenses, the Raspberry Pi running Raspbian saves you ~$70.04 in your first year. That’s a rather definitive advantage to Raspberry Pi.

Schedule

Schedule is always a tough one when it comes to DIY projects. Ultimately, the success and value of these DIY projects rely upon your time being free. That said, there’s a balance between the satisfaction one gets from doing work, and the frustration that comes in when things do not go according to plan. I am going to assume that any frustration is bad, and conversely, lack of frustration is good.

To make this argument, I will compare installing Apache on Windows to installing Apache on a Raspberry Pi. Apache is a web server used by a large number of projects.

Saving myself a good deal of writing in the process, I’m going to link you to the Apache Software Foundation’s instructions to install Apache on Windows. Have a look at this and then come back.

https://httpd.apache.org/docs/2.4/platform/windows.html

Now then, the same install on a Raspberry Pi requires you to type the following into the command line and press <enter>:

sudo apt install apache2 -y

Raspbian is based upon Debian, which is a Linux variant. Linux (and it’s progenitor: Unix) generally depends upon what is called a CLI – Command Line Interface. Using this interface allows us to script various processes, and the example above is one such process. Even if that CLI is intimidating to you, (after all a mouse is more intuitive than a CLI) I think you agree that typing that line three times (to account for typos) is still easier and quicker than following the Windows instructions.

If a project ends up needing even hundreds of commands at the CLI to install and configure all of the packages required, those commands may be scripted into a single file. Here is an example of one such script used to install BrewPi Remix:

https://github.com/brewpi-remix/brewpi-tools-rmx/blob/master/install.sh

Windows allows you to download and install a file with just a few clicks. That script I showed is on GitHub. How does a person used to Windows get that file onto a Raspberry Pi and run it to even begin? Enter the strength of the CLI again:

curl -L install.brewpiremix.com | sudo bash

Entering one command downloads (curl) the file (install.brewpiremix.com) and runs it (pipe through bash) as root (sudo). I’ll admit that’s not as “pretty” as opening a web page and clicking a link, but that’s the part that’s “not as easy” as Windows, and the price we ask you to pay for all the rest.

The example I gave above to install Apache was just a fraction of the work that goes into installing BrewPi Remix. Imagine on Windows if I told you that you need to:

  • Run Windows update and install all patches
  • Install:
    • Git
    • Python3
    • Pip3
    • Arduino
    • Apache2
    • php
    • pyserial
    • psutil
    • simplejson
    • configobj
    • gitpython
    • sentry-sdk

Each of those has its website and install/configure instructions. Linux, on the other hand, has the concept of “repositories,” which are essentially a “one-stop-shop” for applications. MS Windows does not have that construct (although the Windows Store hopes to bring some of that.)

There are methods under Windows, known as bundling, by which one may create an installer to do all that work for you. Because there are no standards or rules about how Windows applications install, each may have it’s own peculiarities and conditions by which you install it. Linux does have standards. If I want to create a bundle for Apache and PHP under Windows, I essentially have to write a program that goes to each website, downloads the most current version, executes the individual installers, and then somehow clicks the right buttons for you.

If I want to do the same work, install Apache and PHP under Linux, I need only to issue a one-line command:

sudo apt install apache2 php -y

To bring this all together: When a component used by a Linux-based Open-Source project changes, the developer generally only needs to change a line or two of text within a script. When a component changes in the Windows ecosphere, the end-user needs to relive all the fun of going to individual websites.

Now, consider when a change in a component breaks things in others. All of the components need to work together. Under Windows, the best you can hope for is someone telling you: “don’t upgrade this yet, it will break other things” Under Linux, I can create dependencies in my scripts, conditional flows, which either use the proper versions together or make a different decision if another version is present.

All of this goes to “Schedule.” Your schedule and mine. You want to spend your time brewing and enjoying beer. You do not want to learn to program.

The purpose of an installer is to quickly and easily take a user from “wants to run” to “running” as quickly, efficiently, and simply as possible. In such a complex system of systems, being able to script this experience, and effectively maintain that script, is one of the most important aspects of a project.

Sadly, not all developers make an effort to use these tools to make your experience easier. That makes me sad because the installer is the very first experience people have with your work. If the installation is a mess, there is a good likelihood that you will lose users because they will just drop the whole thing. If a developer spends some time polishing the install process, there’s a much higher chance you will have people up and running and enjoying your hard work.

Finally, I can find no credible evidence that a person who ignores the user experience during the install process on Linux does any better providing a clean experience on Windows. Simply put: a pig wearing lipstick is not going to kiss any better.

Therefore, and since I’m the one writing, I give the advantage to Linux for complex system integration.

Performance

Once the user pays for a system, once the user installs a system, the user needs to operate that system. To me, that embodies performance.

I started this article with the assumption that there needed to be a separate computer to run one of these projects. I have yet to explain why. Many folks will never get to this part of the article, but hopefully, some will. I’ll explain why, when a person spends $1,000+ on a personal computer, it should not be the one to host the project.

A good deal of brewing-related projects in the ecosphere use what is called a “controller” in order to interface with the physical world. This interface may turn a refrigerator on and off to control fermentation temperature, turn on a burner to raise mash temperature, turn on a pump, or track beer as it is dispensed. That interface requires either a physical connection or what’s known as a service. The physical connection is often the decision point that necessitates a dedicated computer for users. Most people do not have a computer in their garage, den, shed, or basement.

In this case, a service or daemon is a fancy name for a program. That program runs and talks to the controller. It needs to be running while the controller is being used. In many cases, for example, BrewPi Remix, the controller is being used continuously. This necessitates a constantly available interface.

Simply put, a little Raspberry Pi (or any dedicated server) is going to be more consistently available than your home computer that is typically shared for several uses. You might be paying bills, playing games, video chatting with your parents, or playing a first-person shooter. Any of these processes may impact the availability or performance of that service.

One also needs to consider how the application is going to be accessed. As a developer, I have no idea whether my users will be running a Windows, Mac, or Linux desktop. Many users have and depend upon smartphones. This variety of user systems has led to a preponderance of applications that present a web-based user interface. A web browser is ubiquitous and, in many cases, provides a consistent application environment for the developer. The project needs to have a web server accessible by any computer on the network. Having a dedicated computer makes this easier, although admittedly, it’s not impossible to have a web server on nearly any PC. Avoiding conflicts are still a consideration. User PC’s are optimized for the user interface, not to be a server of other PC’s on the network.

Above I’ve presented arguments related to the so-called “separation of duties” theory. If you have one tool doing one job, it can be optimized to do that well. You also won’t have an issue when one of its other jobs necessitates maintenance. Spending $31 on a Raspberry Pi gives you quite a bit of flexibility in this respect.

One of the things I see often is a desire to present the web interface on the Internet. People think it’s cool, and want to show their project to people at work, or maybe follow along while on a family trip. I won’t go into the danger of exposing an endpoint to the Internet here, but rather just assume people will do it no matter what I say. If it’s going to happen, having that access method connect to a small dedicated computer is far better than having the Internet able to access the same computer with credit card information, medical records, and maybe even “those pictures” nobody else knows you have.

Finally, there’s the issue of the development environment. If I intend to present an application to the homebrew community, I need to know upon what they will be running that application. Big companies can afford to develop for two different platforms, but I’m doing this for free. Assuming one and only one environment is probably going to allow me to focus more deeply on the functionality of the application. If I have to choose one, it is more likely to be accepted if I choose one that runs on a $31 computer by itself than to require a Mac user to buy a PC.

Summary

I am one guy. There are exceptions to rules. Everyone is different. Your mileage may vary. Will not treat or prevent sexually transmitted infections. What I have shared here are only some of the arguments, but the ones which guide me. I may not convince you I am 100% right, but hopefully, I will have convinced you there are good reasons for the decisions I and others have made.

Raspberry Pi Wireless Connectivity

The Raspberry Pi is an awesome little platform with uses from education to home automation. If you rely upon it to host a process or application wirelessly, you will sooner or later find it unavailable on the network. If you search the Internet you’ll find several “solutions”, some work a little, some not at all.

I’ve developed a script and a process to keep it running which will shut off power management for the wireless adapter and try to ping the gateway on interval. If the ping fails for a certain number of times, it will reset the adapter. If it continues to fail it can (optionally) reboot the Pi.

Install with the following command:

curl -L wifi.brewpiremix.com | sudo bash

If you wish to enable the reboot function, edit these lines:

# Reboot on failure
REBOOT=false

Add change to:

# Reboot on failure
REBOOT=true

After that change, reboot, or restart the daemon with the following command:

sudo systemctl restart wificheck

The git repository may be found here.

For the Masochists (or the truly paranoid)

(This article is written for BrewPi Legacy Remix version 0.5.2.0, it likely applies to other versions but there may be some differences.)

You’re here for one of a few reasons:

  • You’re just curious about how things work
  • You hate easy things and want to do this the most difficult way possible
  • You are exceeding (perhaps justifiably) paranoid and refuse to run scrips as root. Maybe starting at the bottom of this article is a good place to start?
  • You are one of those people that find some twisted satisfaction in running something named BrewPi on something other than a Raspberry Pi

We’re not here to judge (or am I silently judging you right now?), we’re just here to help. Just like the government. Here’s what you have to go through to replicate most of the scripting in the BPR tool set. Note that this only addresses a single-chamber setup. I’m not sure I feel like describing what has to be done to do multi-chamber but a smart person can figure it out from the information that’s out there for the Legacy BrewPi.

Finally before we dig in, these instructions are written for Debian-based Linux distributions (of which Raspbian is one.) If you use something else, chances are you already understand the differences.

Prerequisites

The bootstrap especially does some checking which may not specifically be strictly required, but is highly recommended:

  • Check to see if the default password of ‘raspberry’ is still current for the ‘pi’ user. If so, prompt to change it.
  • Prompt to set the proper timezone. Since BrewPi works on schedules and the graph is of course time-based, having the proper system time is important. Yes, I could do some work to allow UTC and display in the user time zone … but why?
  • If the hostname is still ‘raspberrypi’, prompt to change it. More than a few folks have more than one Raspberry Pi, and being able to use a nice name like “brewpi.local” to get to the system is handy. Plus, you should never have two machines with the same name on the same network segment. It confuses anything with an ARP table (like your router.)

APT Repositories

Manipulation of APT repositories is required to support this product. This obviously needs to be done as root, however the repos are GPG signed so in theory there’s an added level of security.

As with any other apt packages, it is always good practice to refresh your local apt indexes before doing any installs, updates, or uninstalls. This command will re-synchronize the package index files from their sources. The indexes of available packages are fetched from the location(s) specified in /etc/apt/sources.list.

sudo apt update

There are some packages known to conflict with the solution as well; namely nginx and anything related to php5. This should not be a surprise since we use apache2 as a web server and nginx is a web server itself. Yes a crafty person can make them coexist, if you’re that crafty you don’t need my tutorials. Some of the packages I suggest to reinstall will be reinstalled again with apache2 and that’s fine. The process may also suggest removing other packages which are included by dependency; remove those as well. To uninstall nginx, use the following command:

sudo apt remove libgd-tools, fcgiwrap, nginx-doc, ssl-cert, fontconfig-config, fonts-dejavu-core, libfontconfig1, libgd3, libjbig0, libnginx-mod-http-auth-pam, libnginx-mod-http-dav-ext, libnginx-mod-http-echo, libnginx-mod-http-geoip, libnginx-mod-http-image-filter, libnginx-mod-http-subs-filter, libnginx-mod-http-upstream-fair, libnginx-mod-http-xslt-filter, libnginx-mod-mail, libnginx-mod-stream, libtiff5, libwebp6, libxpm4, libxslt1.1, nginx, nginx-common, nginx-full

To uninstall php5, I could give you a script but you’re here because you either don’t trust my scripts or you can’t use them for some reason. You can use the following command to find php5 packages which may be installed:

dpkg --get-selections | awk '{ print $1 }' | grep 'php5'

You would have to uninstall each of the returned package names with:

sudo apt remove {package name}

Now you need to install a series of APT repos. Here’s the command you would run to install all repos used in this product:

sudo apt get git arduino-core git-core pastebinit build-essential apache2 libapache2-mod-php php-cli php-common php-cgi php php-mbstring python-dev python-pip python-configobj php-xml

Pip Installs Packages (pip)

In addition to the apt packages, several additional python packages will be necessary. These are installed with pip:

 sudo pip install pyserial psutil simplejson configobj gitpython --upgrade 

User Setup

BrewPi runs as its own (now passwordless) user. Here we create the brewpi user and add it to the dialout, sudo and www-data groups:

useradd brewpi -m -G dialout,sudo,www-data

Next we add the ‘pi’ user (or whatever user you normally use to the www-data and brewpi groups:

usermod -a -G www-data,brewpi pi

Clone git Repositories

There’s four repositories that make up this product:

  • BrewPi-Tools-Remix – The toolset that this article replaces, so we won’t clone it here
  • BrewPi-Script-Remix – The core Python scripts which communicate with the Arduino and website
  • BrewPi-WWW-Remix – The PHP website which forms the user interface
  • BrewPi-Firmware-Remix – This repository does not need to be cloned, one need only download the current compiled firmware

Make sure the /home/brewpi directory exists and is empty. Clone the scripts as the brewpi user with the following command:

sudo -u brewpi git clone -b master --single-branch https://github.com/brewpi-remix/brewpi-script-rmx.git /home/brewpi

The web root should be /var/www/html but new versions may change that. Clean out that directory (it probably has a default index.html file there) and clone the website as the www-data user with the following command:

sudo -u www-data git clone -b master --single-branch https://github.com/brewpi-remix/brewpi-www-rmx.git /var/www/html 

Set Permissions

One of the more common solutions to a whole slew of issues is setting the permissions correctly. There is a script for this in:

/home/brewpi/utils/doPerms.sh

If you are still needing or wanting to proceed manually, the following commands are issued:

chown -R www-data:www-data /var/www/html
find /var/www/html -type d -exec chmod 2770 {} \;
find /var/www/html -type f -exec chmod 640 {} \;
find /var/www/html/data -type f -exec chmod 660 {} \;
find /var/www/html -type f -name ".json" -exec chmod 660 {} \;
chown -R brewpi:brewpi /home/brewpi
find /home/brewpi -type d -exec chmod 775 {} \;
find /home/brewpi -type f -exec chmod 660 {} \;
find /home/brewpi -type f -regex "..(py|sh)" -exec chmod 770 {} \;
find /home/brewpi/logs -type f -iname "*.txt" -exec chmod 777 {} \;
find /home/brewpi/settings -type f -exec chmod 664 {} \;

Arduino Firmware

You can download the precompiled firmware (sorry, not going to go into how to compile it on your own here) from the GitHub repo. For instance here’s how to download the latest Arduino Uno firmware:

curl -O https://github.com/brewpi-remix/brewpi-firmware-rmx/releases/download/0.2.10/brewpi-arduino-uno-revC-0_2_10.hex

You would now flash your connected Arduino with the following command. Be sure to update the port (following the ‘-P’) with the port where your Arduino resides:

avrdude -F -e -p atmega328p -c arduino -b 115200 -P /dev/ttyACM1 -U flash:w:"brewpi-arduino-uno-revC-0_2_10.hex" -C /usr/share/arduino/hardware/tools/avrdude.con

Daemon Unit Files

BrewPi and a checker for the WiFi connection run as a systemd daemon. The following files must be placed in the /etc/systemd/system directory. The first is named brewpi.service and contains the following text:

[Unit]
Description=BrewPi service for: brewpi
After=multi-user.target

[Service]
Type=simple
Restart=on-failure
RestartSec=1
User=brewpi
Group=brewpi
ExecStart=/bin/bash /home/brewpi/utils/doBrewPi.sh -d
SyslogIdentifier=brewpi

[Install]
WantedBy=multi-user.target

Change permissions on the file, enable and start the service with the following commands:

sudo chown root:root /etc/systemd/system/brewpi.service
sudo sudo chmod 0644 /etc/systemd/system/brewpi.service
sudo systemctl daemon-reload
sudo systemctl enable brewpi
sudo systemctl start brewpi

The (optional) WiFi check file should be saved as wificheck.service and it contains:

[Unit]
Description=BrewPi service for: wificheck
After=multi-user.target
[Service]
Type=simple
Restart=on-failure
RestartSec=1
User=root
Group=root
ExecStart=/bin/bash /home/brewpi/utils/doWiFi.sh -d
SyslogIdentifier=wificheck
[Install]
WantedBy=multi-user.target

And as with the brewpi daemon, set permissions, install and start it with the following commands:

sudo chown root:root /etc/systemd/system/wificheck.service
sudo sudo chmod 0644 /etc/systemd/system/wificheck.service
sudo systemctl daemon-reload
sudo systemctl enable wificheck
sudo systemctl start wificheck

Work Complete & Final Thoughts

In theory, you should now have a working BrewPi Remix running in single chamber mode. I will however leave you with some final thoughts.

If you are one of those folks who got here and followed these instructions out of some sense of increased security by not running my install scripts I’ve got bad news for you. You are still running scripts from some random person on the Internet and some start as root.

With thanks to the folks at Sandstorm.io, I’m going to paraphrase a few of their sentences:

Using curl|bash or curl|sudo bash certainly seems insecure. To anyone with a basic understanding of Unix, the construction makes it really obvious: This command will give the named web site direct access to your system, with the ability to do anything that you could do. This feels very wrong: We should be able to install software without giving the developers full access to our systems, right?

I would be the first to agree that software you install should not be automatically fully trusted. Unfortunately, however, traditional Unix software is always granted the full authority of the user who runs it (and in the case of a stock Raspbian OS this implies passwordless sudo). When you install software on Linux, no matter what package manager you use, you are giving that software permission to act as you. Most package managers will even execute scripts from the package at install time – as root. So in reality, although curl|bash looks scary, it’s really just laying bare the reality that applies to every popular package manager out there: anything you install can pwn you.

If you wish to install any software without giving it full access to your system, you must install it on a dedicated machine, VM, or (perhaps, with caveats) an isolated user account. In truth it’s more work than a lot of people want to deal with and in my experience people spend a lot more time complaining about it than actually taking steps to be secure.

The reality is this:

  • This is free software and you are invited to not run it if you are that concerned. I’d argue however that even the most paranoid among you have downloaded and run applications from the Internet with no additional security. At best, maybe you scanned it for viruses, but I’d be willing to bet a normal home user would not perform a source-code scan. If you are a person who really does a source-code scan, this is all written in bash, PHP, Python and the firmware in a proprietary C-ish language. It’s all there and all able to be downloaded freely and handled however you wish under the terms of the GNU General Public License.
  • This is intended to be run on dedicated hardware. A Raspberry Pi is really perfect for this and a Pi Zero W is about $20. There’s no reason to try to run this on some other rig on which you do other things that could be compromised by rogue code.
  • The last significant vector at risk here is the network to which the BrewPi machine is attached. If you have IoT devices, surely you have a separate wireless LAN segment for them? Put this there. Tom has a great article about IoT security. You can also choose to not have your BrewPi connected to your home network at all. Sure you lose some functionality, but you can always disconnect the network and just use it with a local keyboard, mouse and monitor.

I’m not making fun of people displaying a healthy dose of paranoia. At last count I am tracking 552 website credential pairs, all using unique highly complex passwords. Yes that’s right, I have over 500 passwords to keep track of. In addition I use two-factor whenever possible. Believe me when I say I am a fan of security. Sometimes though you gotta pick your fights and mitigate risk when you do.

One last word: As a fan of security I do have several ideas how to make this application more secure. I don’t have a schedule and nobody is paying for this so who knows what I’ll get done. Stay tuned though, surely the battle will be won in small skirmishes.

BrewPi Remix FAQ

  • What about some other scenario, when will you test that?” – Maybe never. This is not a commercial venture; chances are once I’m “done enough” making it work on the target system, I’ll be done for good. The original/current BrewPi is a far more capable system, with a wider adoption base, and excellent support. That’s probably a better choice for you if you want to venture from this path I’ve created for you.
  • Do you plan to create/implement/merge {insert cool idea here} functionality?” – No I do not. I’m not a software developer by trade, and this is not a commercial venture so there’s probably little reason to implement something I’ll never use. To be embarrassingly and brutally honest, I hardly get a chance to even brew anymore. I started this initially to make it easier for a friend of mine to get going again after his Pi ate his SD card. I’ll repeat: The original/current BrewPi is a far more capable system, with a wider adoption base, and excellent support. That’s probably a better choice for you if you want expanded capabilities.
  • Will you accept pull requests?” – Maybe. Here’s the honest truth however: Not being a software developer by trade means that working with typical software development tools in a collaborative environment like GitHub is new to me. I am probably doing this wrong/poorly and in a way that doesn’t easily allow such collaboration. If you’re willing to work with someone who does not have these skills in order that you may contribute your own work, it’s likely best to contact me directly before you start so we can work out the details to avoid frustration for both of us (mostly you.)
  • What about older versions of the Pi or Raspbian Stretch, etc.?” – I’ve no reason to believe older versions will not work, but they’ve not been tested. In theory it should work fine, but at some point, on a platform like Raspberry Pi, you just need to say “flash a new card and get over it.” These are not desktop machines that accumulate “stuff” over the years. If you have a Pi that’s on it’s original SD card for more than a couple years you have a rare bird indeed. I’d be more than happy to discuss why it didn’t work if you run into an issue, it would be interesting I think, but it might not be something I choose to address.

Back to Installation Instructions.

Security Note

My instructions tell you to copy and paste a command into your terminal window. Despite me telling you to do that, I am now going to tell you how unsafe that is. Many people browse the Internet, find the command they need, and blindly paste it into their terminal window. This one is blatantly (potentially) dangerous from a non-trusted source:

wget -qO- https://u.nu/brewpi-tools-remix | sudo bash

It’s going to download a script to your Raspberry Pi, and pipe (|) it through the command sudo bash. When you use sudo without any other arguments it will run the command which follows with root privileges. So, you basically found someone on the Internet telling you to run their code as root, without even knowing what it all does. Despite the inherent risk, installing an application as root is often necessary since some applications have to make global changes to your system.

This is how bad things happen.

Even if you think you completely understand the command you are reading and copying, there is still an opportunity for a specially crafted web page to make the command look like one thing, but be a completely different command when you paste it. That would be A Bad Thing™. For an example, copy and paste this code into your terminal (or if you are nor suitable paranoid, into a text editor:

git clone /dev/null; clear; echo -n "Hello ";whoami|tr -d '\n';echo -e '!\nThat was a bad idea. Don'"'"'t copy code from websites you don'"'"'t trust!
Here'"'"'s the first line of your /etc/passwd: ';head -n1 /etc/passwd
git clone
git://git.kernel.org/pub/scm/utils/kup/kup.git

(From this page which describes this copy/paste vulnerability.)

The lesson to be learned from this is if you are going to copy/paste a command from any source, always use an interim paste into a text editor like Notepad to make sure A Bad Thing doesn’t happen to you. Now you can join your previously scheduled show: Install Instructions, which is patiently waiting for you where you left it.

Wait. Now you don’t know if you should trust the setup command I provided? I’m shedding a happy tear. Security and the Internet is a rabbit hole filled with (justifiable) paranoia and bad actors. Your choices here however are:

  1. Trust me and run it
  2. Examine that script carefully and make sure it does nothing bad. Then, since the first one executes as root you need to follow that to the next one because it inherits that security construct. Ultimately you can drive yourself crazy when you realize the implications, or just accept that whenever you install free code from the Internet you take your chances. This is the case with any software, not just BrewPi.
  3. Do a manual install. I’ve detailed how that might be done in a post called: “For the Masochists.”

At some point I hope to list out the manual install steps necessary to replicate this series of automated steps for the truly paranoid. That’s quite frankly not a high priority for me since the goal here is to make BrewPi Legacy Remix available to every-day people. Those of you who would choose the manual steps can probably figure it all out anyway.

Cheers!

Return to the Installation Instructions.

Project Assumptions and Proceedings

This tool set adds a bootstrap to install the BrewPi Legacy Remix packages on a completely fresh install of Raspbian (codename “Stretch” at the time of this writing). I have created this bootstrap because some steps required in previous iterations were a little alien to people new to Raspbian/Linux. Additionally, some supporting software has been deprecated/upgraded which before now made the older BrewPi packages incompatible.

This bootstrap will:

  • Check a few things
  • Handle some Raspberry Pi setup steps if/as needed or recommended
  • Install some supporting files
  • Download and execute the BrewPi-Tools-RMX installation scripts
  • Perform some final cleanup

In order to make this work well, I have to make some assumptions about the environment in which this will be run. Here I’ll try to list some, however I am sure someone will find a way to try something I’ve not considered. Do not over-think this. Don’t fiddle around with your Pi before running the bootstrap. Turn it on, connect to your home network, and go. Here’s a list of known (or at least recalled) assumptions made during this project:

  • This has been developed and tested on a Raspberry Pi 3 B+ because that’s what I have laying around. I have absolutely no reason to believe it would not work on a Zero, 2B, or other versions of the Raspberry Pi line. I’ve just not tested it.
  • This has been developed and tested on the Raspbian OS. Raspbian is based on Debian, so using a Debian (or derivative) OS distribution may work. However, that’s not been tested. I am not at all sure that it would work on a different flavor of Linux.
  • This has been developed and tested on the Raspbian Stretch distribution. If a new distribution for the Raspberry Pi is released it may no longer work. I hope I’ve future-proofed it, however the original/core code may have some non future-proofed areas waiting to rear their ugly head (or I may not be as good at future-proofing as I believe.)
  • I’ve assumed throughout that this is the only function the Pi will handle. This is not unique to BrewPi in general. I’ve not specifically prevented it form doing anything else, I just can’t (and won’t try) to test every permutation.
  • This will not create a BrewPi which is secure enough to connect to from the Internet. There’s a whole host of reasons for this, but please, do not do it unless you know what you are doing. I suggest you consider Dataplicity if you really need/want to do this.
  • This has been developed and tested using the default user ‘pi’ which by default has password-less sudo rights. This is how Raspbian is shipped, and this is how I’ll continue to test it. If you know enough to change any of these assumptions, you know enough to figure out why this process may not work for you. If you simply MUST change this, I suggest you do it after you get BrewPi Remix running.
  • You need for your Pi to have access to the Internet. I think this is obvious, but the Pi needs to access GitHub and standard Raspbian repositories to download code. Generally speaking, plugging your Pi into your home network with an Ethernet cable will do this without any configuration necessary. Attaching to wireless will take a little more work that’s not in scope of this project specifically, but for which you will find some help in “Headless Raspberry Pi“.
  • This has been developed and tested on a bone-stock Raspbian setup, with no user or local customization implemented. The only things that has been tested which do not inherently work on a fresh setup is wireless connectivity and ssh over wireless. The bootstrap script will:
    1. Check to make sure the script has executed with sudo to root (this is how the instructions will work if you follow them)
    2. Provide some rudimentary instructions
    3. Check for default password, and prompt to change it if so
    4. Set the proper timezone
    5. Prompt to optionally change the host name if it is currently the default ‘raspberrypi’
    6. Check network connectivity to GitHub (this part should be a given since it’s intended to be run via wget but I’m not going to assume someone can’t break my plans)
    7. Run an apt update if it’s not been run within the last week
    8. Install git packages via apt get to allow the rest of the install to work
    9. Install Python packages via pip.
    10. Clone the BrewPi Tools RMX into the ~/brewpi-tools-rmx folder
    11. Execute install.sh which is responsible for the rest of the setup

I am certain that someone will find an important assumption I did not list here. We’ll see how long that takes. Let me know what you find.

Return to Installation Instructions.