Internet Installs

2020-11-21 by Rev. Fr. Robert Bower

After reading several stories about Internet installs, I thought I would write a post on the best way to prepare for an Internet install.

No matter your provider or the type of Internet you are going to receive there are things you need to keep in mind.

Ask questions about your install. One of the best places is your provider's forum.
Know what your provider is willing to do during an install and what they will not do. For example, your provider will not run cabling in attics, crawl spaces, or walls.
Make a plan on what you need to do and what the provider's installer will do.

Let me give you an example of my recent AT&T Fiber Optic install.

Before I ever called AT&T I posted on the forum what I needed to have done and if the AT&T installer would do it or if I needed to do some of it myself or have a third-party do some of the work.

I wanted the new fiber optic line ran from the outdoor patch panel through the attic and through the wall of my office. I was quickly told AT&T would not do any work in the attic or fish wires through the wall. I was given the suggestion of running conduit from where I wanted the line to enter the house to where I wanted the line to enter my office, then run a string through conduit for a pull line. This way the AT&T installer could connect the fiber optic line to the string and pull the line through the conduit from the other end using the string.

My days of climbing in an attic are long gone so I had my electrician run the conduit for me. He installed a lightweight plastic conduit. The conduit is called "Smurf Tube" due to its blue color. He also fished a string through the conduit so the installer could pull the fiber optic line through the conduit.

When the day of my install came everything went great. The installer installed a patch panel to the side of the house, ran the fiber line from the pole to the patch panel using the existing hangers, pulled the optic line from my office out to the patch panel, installed the Optic Network Terminal, and connected it to AT&T Gateway. He than powered everything up and after a few updates to the gateway everything was working great.

By planning ahead and getting some great advice I was able to have a quick and smooth install. If I had not plan ahead I would have had to either reschedule the install or have the gateway installed in a different location that will not work as well.

One other thing I would mention is don't make WIFI the primary way to connect to the Internet. WIFI is second rate to a connection using an Ethernet cable. There are so many things that limit and interfere with your WIFI too many to talk about here. Just know that you signal will never be as strong, fast, or have the range as you think it should be.

Zettelkasten ID gettingreadyforintenetinstall-2020-11-21-1224

Firefox, Firejail, and Hosts

2020-09-21 by Rev. Fr. Robert Bower

There is always going to be struggle for safe untracked browsing on the Internet. I am not sure it is completely possible, but we can try to make ourselves as safe and allusive as possible. I am no security expert so take this all with a grain of salt.

While there are many tools to try and accomplish this I have settled in on three tools to make surfing as safe as possible and keep my profile to myself. The three tools are Firejail, the use of profiles with Firefox, and my hosts file.

The first tool I want to talk about is Firejail. It can sandbox any program you run with it. The program's description from its web page, "Firejail is a SUID program that reduces the risk of security breaches by restricting the running environment of untrusted applications using Linux namespaces and seccomp-bpf. It allows a process and all its descendants to have their own private view of the globally shared kernel resources, such as the network stack, process table, mount table."

So if you run Firefox within Firejail Firefox has a limited ability to read your system and even more limited ability to write to your system. The default Firejail profile for Firefox limits Firefox write permissions to the Downloads directory and writing to your Firefox profile. This limits the amount of damage a malicious website can do to your system. For added security, if you use the private switch in Firejail anything from the session is deleted when the session is closed.

I use Firefox profiles in conjunction with Firejail to help prevent trackers from building a profile of me. I have a Firefox profiles for Google, the School I teach at, banking, Amazon, and an unknown profile. Profiles themselves do nothing to prevent you from being tracked. It is how profiles are used that help prevent trackers for building a complete profile. By using multiple Firefox profiles you look like multiple users to the outside world.

I start each Firefox profile in its own instance of Firejail so each profile is isolated from one another. Your behavior is what makes this work. In my Google profile I only use Google services. In my banking profile I only do banking. Again this doesn't limit tracking, it only makes you look like multiple users. So by having multiple profiles for different tasks, hopefully I am a little harder to profile. FYI, it takes some practice browsing with multiple profiles, but it gets easier with some practice. I use the I3 window manager, so if my browser windows are in tab mode it is like browsing with tabs that have subtabs.

The last tool I have does help limit tracking and that is my hosts file. A host file is like a bit of on board DNS. Usually it has that 127.0.0.1 is local host and a few other things but one thing you can do is add websites and corresponding ip addresses to it, so DNS requests are not made. For example a large domain may have multiple ip addresses but if you always want to connect to a certain ip address you could specify the ip address in the hosts file.

You can also use it to block ads and trackers. Lets say you go to a web page example.com but it has an ad tracker embedded that loads ads from anoyingads.com.

In your hosts file you can enter

0.0.0.0 annoyingads.com

What happens is Firefox rather checking the DNS server for annoyingads.com ip address it just uses the one in the host file. Since 0.0.0.0 signifies a non-routable address the ad / tracker from annoyingads.com doesn't load. You can create your own list but there are many premade lists. The best hosts file that blocks ads and trackers is at https://github.com/StevenBlack/hosts It is a combination of several lists. Just add the list to your existing hosts file and many of those trackers and ads will not show up in your browser.

Again, I am not a security expert just a guy trying to browse a little safer than the day before.

Zettelkasten ID firefoxandfirejail-2020-09-20-1517

Newsboat meets Readability CLI

2020-08-14 by Rev. Fr. Robert Bower

Newsboat is my RSS feed reader. It allows me to check the websites I am interested in quickly and without distraction. I have been able to create macros that keep me on the command line from videos to mp3s to actual articles. There is one small glitch in all this, sites that only include a description of the articles in the RSS feed and force you to go to the website to read the whole article.

At this point there are two options. The first being using a command line web browser to read the article and scrolling through pages of menus and ads to find the actual article. The second being leaving the command line and reading the article in Firefox's reader mode.

Now there is a third option using Readability CLI to read articles in "Firefox's reader mode" on the command line.

Readability CLI by gardenappl using Mozilla's Readability library to strip a web page to the core content. The result being a simplified HTML document which can be viewed by any browser.

I created a two line bash script that is called by a Newsboat macro key that ties together Newsboat, Readability CLI, and Lynx.

#!/bin/sh  
readable $1 > /tmp/readablearticle.html
uxterm -e lynx /tmp/readablearticle.html

The first line after the #!/bin/sh has readable, the executable for Readability CLI download the url provided by Newsboat, convert it to, using Mozilla's Readability library, a HTML file with just the core content, and finally save the new file in the tmp directory. The second line opens an xterm terminal and executes the opening of the HTML file with Lynx.

Readability CLI can be found at GitLab

A Video, by Brodie Robertson, of Readability CLI in action

Zettelkasten ID newsboatmeatsreadable-2020-08-14-1545

Lynx Configuration Five Ways

2020-07-04 by Rev. Fr. Robert Bower

When it comes to terminal based web browsers I only know of two that are actively maintained. Today I am going to write about one of those, Lynx. Lynx can be very frustrating to the new user or maybe it was just me. What I found frustrating about Lynx was how to configure it. Lynx actually is not hard to configure once you know that Lynx has five different methods to configure it with some of the methods overlapping but many options being unique to a method. None of the methods are difficult to do. It just knowing that different things are configured in different places. I am not going to show you how to configure Lynx but what the five configuration methods are and where to find more information about them in the manual.

The five methods are:

lynx.cfg
.lynxrc
lynx.lss
Command Line options
Environment Variables

Lets summarize at each method.

.lynxrc

The .lynxrc configuration file holds the options found in the Options menu that can saved from session to session. Remember not all options can be saved from session to sessions using the Options menu. More information about the Options Menu and .lynxrc can be found here

lynx.cfg

The lynx.cfg configuration holds most of the other configurations for Lynx such as the start page and the index page. When you install Lynx depending on your distribution there may only be a system wide lynx.cfg. I would suggest copying the file to your home directory and then customizing the file to fit your needs. Details of the lynx.cfg file can be found here

lynx.lss

The lynx.lss configuration file holds the style settings for Lynx, such as color. Again, when you install Lynx depending on your distribution there may only be a system wide lynx.lss. I would suggest copying the file to your home directory and then customizing the file to fit your needs. The best way to learn about lynx.lss is just play with the file. If you break it just copy it again from the system wide file.

Environment Variables

You can set environment variables for Lynx so it knows where to look for items as it starts up. For example, Lynx looks it the root of the user's home directory for lynx.cfg and lynx.lss but lets say you want to follow the current standard of putting configuration files in the /home/user/.config/lynx directory. You can set add environment variables to change that path in your .zshenv file so Lynx looks for the configuration files in a different location.

An example being:

export LYNX_CFG=/home/yourusename/.config/lynx/lynx.cfg
export LYNX_LSS=/home/yourusername/.config/lynx/lynx.lss

More information can be found here

Lynx command line

If you want to start Lynx with certain options set that can not be set by one of the configuration files or you want to just change something for one session Lynx has many different command line options.

The two I use the most -useragent= and -cookies. The -useragent= flag lets me set user agent string whatever I want. Usually I change it from lynx to l_y_n_x so that I can go to sites that usually block the Lynx browser.

I also use -cookies if I want cookies set to ignore.

More information can be found here

Gotchas

There are a few gotchas in Lynx that makes configuration difficult until you know about them.

One is Lynx assumes your bookmark files and cookies files will be in the root of your home directory and prepends the path with /home/yourusername/ for paths listed in your .lynxrc file.

For example, lets say you want to you want to put the bookmarks file in your .local/share/lynx directory. In the .lynxrc file you would only put bookmark_file=.local/share/lynx/lynx_bookmarks.html Lynx will prepend the path with /home/username/

Another gotcha is in the Options menu. Any option ending in a (!) can only be changed for that session only using the option menu. A user can change the user agent in the options menu but once you quit Lynx the change is gone.

Hopefully this gives you a simple breakdown of where Lynx looks for different customization information.

Zettelkasten ID lynxconfiguration-2020-07-04-1143

Using MPV with Newsboat

2020-05-27 by Rev. Fr. Robert Bower

I use Newsboat to read RSS feeds including mp3s and YouTube Videos. I want to be able to watch YouTube videos from the command line and not from the browser. The standard suggestion is to create a macro in Newsboat and pass the YouTube url to mpv. MPV then takes the url and plays the video. Here is where I was having issues. The URL you see in your browser for the YouTube video is not actually the URL for the video that gets played on your computer. The URL you see in your browser is the "doorway" to the actual video URLs. For each video there are several URLs, ones that are video only, ones that are audio only, ones that are high quality, and ones that are low quality. I find that sometimes, not always, MPV has trouble finding an URL to play. Sometimes it will work fine, sometimes it will play the audio only URL, and sometimes it cannotnot find a URL to play at all. This problem occurs even with the most up to date youtube.lua file for MPV.

The one program I find that is very good at keeping up with the changes at YouTube is youtube-dl. It is a command line program that lets you download YouTube videos. YouTube-dl can be used to download the desired video and then the video can be watched with MPV. But what if you want to stream the video and not download it. YouTube-dl has a neat little flag -g which gets the video's URLs but doesn't download them. Youtube-dl also has a -f flag which allows the user to specify what format they want. One of the options is best. So if you use youtube-dl -g -f best http://youtube.com/watch?=videoidgoeshere youtube-dl will send to stdout the URL of the best quality video. This URL MPV can play without any issues.

So here come a bash script to the rescue:

 #!/bin/sh  
 youtube-dl -g -f best $1 | xargs mpv

This little script has one argument, the URL of the video's YouTube page. Youtube-dl finds the URL of the best quality video and then it is piped to MPV and the video is played.

To call this from Newsboat you need to create a macro.

 macro m set browser "tsp nbyt %u"; open-in-browser ; set browser "lynx"

The macro sets the browser to the script I created called nbyt, then opens the script while passing the URL to it, and finally sets the browser back to the default. The tsp command allows the script to be run in the background so Newsboat is able to do something else.

One key to keep this all working is to keep youtube-dl updated with youtube-dl -U. YouTube is constantly changing things so youtube-dl must be constantly updated so it continues to work.

As always feel free to use the script at your own risk.

Zettelkasten ID mpvyoutube-2020-05-27-0831