Virtual environments are really useful for maintaining different packages and for separating different environments without getting your system messy. In this post I will go over some of the various virtual environment tricks that I seem to always forget if I haven’t worked with Python in awhile.
This post is meant to be mostly a reference for remembering commands and syntax and other helpful notes. I’d also like to mention that these steps were all tested on OSX, I haven’t tried on Windows so don’t know if it is any different.
Working with virtual environments
There are a few pieces in order to get to get started. First, the default version of Python that ships with OSX is 2.7, which is slowly moving towards extinction. Unfortunately, it isn’t exactly obvious how to replace this version of Python on OSX.
Just doing a “brew install python” won’t actually point the system at the newly installed version. In order to get Python 3.x working correctly, you need to update the system path and place Python3 there.
You will want to put the above line into your bashrc or zshrc (or whatever shell profile you use) to get the brew installed Python onto your system path by default.
Another thing I discovered – in Python 3 there is a built in command for creating virtual environments, which alleviates the need to install the virtualenv package.
Here is the command in Python 3 the command to create a new virtual environment.
python -m venv test
Once the environment has been created, it is very similar to virtualenv. To use the environment, just source it.
source test/bin/activate
To deactivate the environment just use the “deactivate” command, like you would in virutalenv.
The virtualenv package
If you like the old school way of doing virtual environments you can still use the virtualenv package for managing your environments.
Once you have the correct version of Python, you will want to install the virtualenv package on your system globally in order to work with virtual environments.
sudo pip install virtualenvwrapper
You can check to see if the package was installed correctly by running virtualenv -h. There are a number of useful virtualenv commands listed below for working with the environments.
Make a new virtual env
mkvirtualenv test-env
Work on a virtual env
workon test-env
Stop working on a virtual env
(when env is actiave) deactive
List environments
lsvirtualenv
Remove a virtual environment
rmvirtualenv test-env
Create virtualenv with specific version of python
mkvirtualenv -p $(which pypy) test2-env
Look at where environments are stored
ls ~/.virtualenvs
I’ll leave it here for now. The commands and tricks I (re)discovered were enough to get me back to being productive with virtual environments. If you have some other tips or tricks feel free to let me know. I will update this post if I find anything else that is noteworthy.
There is an annoying behavior that you might run across in the Windows version of Vscode if you are using the Vsvim plugin when trying to use the default hotkey <ctrl+b> to toggle the sidebar open and closed. As far as I can tell, this glitch doesn’t exist in the OSX version of Vscode.
In vim, the shortcut for this toggle is actually used to scroll the page buffer up one screen. Obviously this makes behavior is the default for Vim but it also is annoying to not be able to open and close the sidebar. The solution is a simple hotkey remap for the keyboard shortcut in Vscode.
To do this, pull open the keyboard shortcuts by either navigating to File -> Preferences -> Keyboard shortcuts
Or by using the command pallet (ctrl + shift + p) and searching for “keyboard shortcut”.
Once the keyboard shortcuts have been pulled open just do a search for “sidebar” and you should see the default key binding.
Just click in the “Toggle Side Bar Visibility” box and remap the keybinding to Ctrl+Shift+B (since that doesn’t get used by anything by default in Vim). This menu is also really useful for just discovering all of the different keyboard shortcuts in Vscode. I’m not really a fan of going crazy and totally customizing hotkeys in Vscode just because it makes support much harder but in this case it was easy enough and works really well with my workflow.
Kubernetes is complicated, as you’ve probably already discovered if you’ve used Kubernetes before. Likewise, the Kubectl command line tool can pretty much do anything but can feel cumbersome, clunky and generally overwhelming for those that are new to the Kubernetes ecosystem. In this post I want to take some time to describe a few of the CLI tools that I have discovered that help ease the pain of working with and managing Kubernetes from the command line.
There are many more tools out there and the list keeps growing, so I will probably revisit this post in the future to add more cool stuff as the community continues to grow and evolve.
Where to find projects?
As a side note, there are a few places to check for tools and projects. The first is the CNCF Cloud Native Landscape. This site aims to keep track of all the various different projects in the Cloud/Kubernetes world. An entire post could be written about all of the features and and filters but at the highest level it is useful for exploring and discovering all the different evolving projects. Make sure to check out the filtering capabilities.
The other project I have found to be extremely useful for finding different projects is the awesome-kubernetes repo on Github. I found a number of tools mentioned in this post because of the awesome-kubernetes project. There is some overlap between the Cloud Native Landscape and awesome-kubernetes but they mostly compliment each other very nicely. For example, awesome-kubernetes has a lot more resources for working with Kubernetes and a lot of the smalller projects and utilities that haven’t made it into the Cloud Native Landscape. Definitely check this project out if you’re looking to explore more of the Kubernetes ecosystem.
Kubectl tricks
These are various little tidbits that I have found to help boost my productivity from the CLI.
Tab completion – The first thing you will probably want to get working when starting. There are just too many options to memorize and tab completion provides a nice way to look through all of the various commands when learning how Kubernetes works. To install (on OS X) run the following command.
brew install bash-completion
In zsh, adding the completion is as simple as running source <(kubectl completion bash). The same behavior can be accomplished in zsh using source <(kubectl completion zsh).
Aliases and shortcuts – One distinct flavor of Kubernetes is how cumbersome the CLI can be. If you use Zsh and something like oh-my-zsh, there is a default set of aliases that work pretty well, which you can find here. There are a many posts about aliases out there already so I won’t go into too much detail about them. I will say though that aliasing k to kubectl is one of the best time savers I have found so far. Just add the following snippet to your bash/zsh profile for maximum glory.
alias k=kubectl
kubectl –export – This is a nice hidden feature that basically allows users to switch Kubernetes from imperative (create) to declarative (apply). The --export flag will basically take an existing object and strip out unwanted/unneeded metadata like statuses and timestamps and present a clear version of what’s running, which can then be exported to a file and applied to the cluster. The biggest advantage of using declarative configs is the ability to mange and maintain them in git repos.
kubectl top – In newer versions, there is the top command, which gives a high level overview of CPU and memory utilization in the cluster. Utilization can be filtered at the node level as well as the pod level to give a very quick and dirty view into potential bottlenecks in the cluster. In older versions, Heapster needs to be installed for this functionaliity to work correctly, and in newer versions needs metrics-server to be running.
kubectl explain – This is a utility built in to Kubectl that basically provides a man page for what each Kubernetes resource does. It is a simple way to explore Kubernetes without leaving the terminal
This is an amazing little utility for quickly moving between Kubernetes contexts and namespaces. Once you start working with multiple different Kubernetes clusters, you notice how cumbersome it is to switch between environments and namespaces. Kubectx solves this problem by providing a quick and easy way to see what environments and namespaces a user is currently in and also quickly switch between them. I haven’t had any issues with this tool and it is quickly becoming one of my favorites.
Dealing with log output using Kubectl is a bit of a chore. Stern (and similarly kail) offer a much nicer user experience when dealing with logs. These tools allow users the ability to do things like show logs for multiple containers in pod, use regex matching to tail logs for specific containers, give nice colored output for distinguishing between logs, filter logs by namespaces and a bunch of other nice features.
Obviously for a full setup, using an aggregated/centralized logging solution with something like Fluenctd or Logstash would be more ideal, but for examining logs in a pinch, these tools do a great job and are among my favorites. As an added bonus, I don’t have to copy/paste log names any more.
yq is a nice little command line tool for parsing yaml files, which works in a similar way to the venerable jq. Parsing, reading, updating yaml can sometimes be tricky and this tool is a great and lightweight way to manipulate configurations. This tool is especially useful for things like CI/CD where a tag or version might change that is nested deep inside yaml.
There is also the lesser known jsonpath option that allows you to interact with the json version of a Kubernetes object, baked into kubectl. This feature is definitely less powerful than jq/yq but works well when you don’t want to overcomplicate things. Below you can see we can use it to quickly grab the name of an object.
kubectl get pods -o=jsonpath='{.items[0].metadata.name}'
Working with yaml and json for configuration in general seems to be an emerging pattern for almost all of the new projects. It is definitely worth learning a few tools like yq and jq to get better at parsing and manipulating data using these tools.
Similar to the above, ksonnet and jsonnet are basically templating tools for working with Kubernetes and json objects. These two tools work nicely for managing Kubernetes manifests and make a great fit for automating deployments, etc. with CI/CD.
ksonnet and jsonnet are gaining popularity because of their ease of use and simplicity compared to a tool like Helm, which also does templating but needs a system level permission pod running in the Kubernetes cluster. Jsonnet is all client side, which removes the added attack vector but still provides users with a lot of flexibility for creating and managing configs that a templating language provides.
More random Kubernetes tricks
Since 1.10, kubectl has the ability to port forward to resource name rather than just a pod. So instead of looking up pods that are running and connecting to one all the time, you can just grab the service name or deployment and just port forward to it.
port-forward TYPE/NAME [LOCAL_PORT:]REMOTE_PORT
k port-forward deployment/mydeployment 5000:6000
New in 1.11, which will be dropping soonish, there is a top level command called api-resource, which allows users to view and interact with API objects. This will be a nice troubleshooting tool to have if for example you are wanting to see what kinds of objects are in a namespace. The following command will show you these objects.
k api-resources --verbs=list --namespace -o name | xargs -n 1 kubectl get -o name -n foo
Another handy trick is the ability to grab a base64 string and decode it on the fly. This is useful when you are working with secrets and need to quickly look at what’s in the secret. You can adapt the following command to accomplish this (make sure you have jq installed).
k get secret my-secret --namespace default -o json | jq -r '.data | .["secret-field"]' | base64 --decode
Just replace .["secret-field"] to use your own field.
UPDATE: I just recently discovered a simple command line tool for decoding base64 on the fly called Kubernetes Secret Decode (ksd for short). This tool looks for base64 and renders it out for you automatically so you don’t have to worry about screwing around with jq and base64 to extract data out when you want to look at a secret.
k get secret my-secret --namespace default -o json | ksd
That command is much cleaner and easier to use. This utility is a Go app and there are binaries for it on the releases page, just download it and put it in your path and you are good to go.
Conclusion
The Kubernetes ecosystem is a vast world, and it only continues to grow and evolve. There are many more kubectl use cases and community to tools that I haven’t discovered yet. Feel free to let me know any other kubectl tricks you know of, and I will update them here.
I would love to grow this list over time as I get more acquainted with Kubernetes and its different tools.
As part of my reflection over the past year, one item I decided to focus on was getting better at goal setting and following through with goals. As a result, I decided to do some research on tools to help organize tasks. I have been using Kanban style boards for organizing work for many years now, so thought it might be a good fit for organizing my personal life. And so far, it has been great. There is a section later in this point that has more details on my current at home Trello flow if anyone is interested.
As I started down this path, I noticed that many areas of organization and time management skills are commonly neglected, both in work and in day to day life at home, so thought I’d share some other tricks I have learned along the way. Keeping on top of the surrounding chaos is a skill that is acquired, and therefore is something that needs to be worked at regularly. Just like getting better at anything else, the process requires constant feeding and attention. My own workflow and style is still constantly evolving, as I still try new tools and techniques regularly. As always, if you have any good advice or useful tools please feel free to share them.
In addition to my Trello discovery, this post will mostly be a brain dump of some of my thoughts and ideas about the processes and tools that I have come to enjoy using in my daily planning, various workflows, and just my overall approach to time management and tasks. I realize that everyone is different, with their own styles for breaking down work and organizing their thoughts and ideas so don’t take this post as the best way to do any one thing. I just want to point out things that I have found to work over the years and hopefully people can take bits and pieces that fit well into their current way of managing the chaos.
For whatever reason, I have always enjoyed organization, attention to detail and meticulous planning. As such, I have read quite a bit about organization and prioritization, including the book that really kick started my interest in getting organized professionally – Time Management for System Administrators: Stop Working Late and Start Working Smart, which I highly recommend.
The conclusion I have come to is that there isn’t one right way to do things for everybody. You have to experiment with different approaches until you find a set of tools and workflows that suits your tastes and makes your more productive. Most importantly, if somebody already has a tool they really like, don’t force them to change the way they do things. Instead, re-fit your workflow to accommodate.
Connecting the dots
The most important aspect I have found for creating your own style of organization is to experiment and just use what works best for you. The tools and apps I mention in this post all work well for me. As an example, one thing that is important to me is that I can access all of my tools, notes, tasks, whatever easily across different devices. Having the ability to view Trello boards is great when I’m on the go but need a quick reference for something I’m working on. Likewise, I pretty much only use Keep to track and update my to-do list items or add new events to calendars, but these tools are also available on any computer I use if I need to look at something quickly or update a list.
A number of these tools also integrate with each other, which helps further improve organization. For example, Google apps and Trello integrate nicely, so Google calendars can be configured to keep track of various Trello boards. Trello can also hook into Google Drive so that online documents can be attached to cards. There are a number of other Trello integrations which I haven’t explored yet but look useful, including Zapier, Dropbox, Slack, Outlook, Gmail, etc. Since Trello is so flexible it can be tailored to work with other tools and workflows.
Simple to-do lists
Simple lists for keeping track of lists and small daily tasks has become invaluable to helping me stay on top of my responsibilities. I keep these quick lists on my phone in Google Keep and none of them usually ever have more than 5 items at a time. If I see that the task is more complicated I will move it into something that is easier to organize and prioritize. For example, I might add a to-do item for fixing the dishwasher. I have discovered that lists can quickly get cluttered and become messy, which will basically work against you and make you not want to use them.
Therefore, if any task or item in my to-do list takes more than a few hours of messing with and/or parts need to get ordered or prioritized or anything that can get more complicated and take more than a day to do, I will move it off the freshly minted to-do list into Trello.
Trello
As described above, Trello is a great tool for organizing a variety of different activities. Trello is flexible so can be used for any number of use cases you can imagine, which allows for some great use cases. As I said, my main objective for using Trello was to more effectively track my goals and tasks as well as those of my family.
In the past I have tried to use documentation and note taking apps to organize and prioritize things I would categorize as “projects” but it has just never stuck very well. Trello really adds another dimension to organization and prioritization of tasks that you just can’t get with something like OneNote, even though I love OneNote. These tools just slightly overlap I would say and Trello is just much better suited towards driving work.
My board relies heavily on labels for doing different kinds of filtering between goals and tasks. If I want my wife to know what’s going on with trip planning for example, I can add a label for her and I can also add her as a subscriber so that she can easily receive updates/changes via email, or if she wants to look at all the progress just open up the card (the mobile version is really good at this).
Notes, attachments, lists, etc, all are being used so I am least leveraging some of the other features that make Trello useful.
All said and done, I am very happy with how easy Trello is to use and how easy it is to track progress of my various happenings. The board I came up with is fairly simple, but I’m looking forward to getting better at it and adding “features” in the future. I know that Trello can do much more for project and task planning so I will be looking into beefing up my board with some of the integrations. For now, the bones are there at least.
OneNote
I have written about OneNote and technical documention in the past so won’t cover it in a lot of detail here. Suffice it to say though, I use this tool on pretty much a daily basis. Pretty much everything I write down goes into OneNote. Including all of my notes, documentation, thoughts, journals, etc. Any time I have an idea or take a note, it goes into OneNote. I like to have separate notebooks for organizing anything from my personal journal to notes about finance and investing to technical documentation and useful links in my daily work.
OneNote is great at handling things likes screenshots, checklists
One word of wisdom if you are thinking of using a note taking app for organizing notes. Your notes really need an INTENSE amount love and care to make them useful and easy to reference, which means constantly updating them, moving pages around, as well as deleting and merging pages that are no longer relevant. The notes are a living and breathing document and are pretty much in a constant state of change. If you don’t keep your thoughts organized and up to date then things will quickly get unwieldy.
I have used Evernote in the past but strongly prefer OneNote because it matches my organization style more closely and the sharing and syncing features fit well with all my other workflows.
Google Calendar
Getting a good calendar app to keep track of daily time and date commitments is crucial.
I really like Google calendar because it is easy to use, is very portable and cross platform. For me, everything that has a due date goes into Google calendar. As an added bonus I can easily share my calendar out to family members as well (assuming they also use Google) and can view their calendars which allows everyone to stay in sync that needs to and helps tremendously with keeping on top of things that are going on, day to day.
There is a fantastic section that covers the topic of using a calendar to stay organized in the Time Management for Systems Administrators book, which I highly recommend if you aren’t really sure how or where to get started with organizing your own calendar.
Honorable mentions
Here are some of the other tools I have found useful as organization and productivity tools but don’t really fit well as time management tools. If you are interested, feel free to check them out.
Bash is great. As I have discovered over the years, Bash contains many different layers, like a good movie or a fine wine. It is fun to explore and expose these different layers and find uses for them. As my experience level has increased, I have (slowly) uncovered a number of these features of Bash that make life easier and worked to incorporate them in different ways into my own workflows and use them within my own style.
The great thing about fine arts, Bash included, is that there are so many nuances and for Bash, a huge number of features and uses, which makes the learning process that much more fun.
It does take a lot of time and practice to get used to the syntax and to become effective with these shortcuts. I use this page as a reference whenever I think of something that sounds like it would be useful and could save time in a script or a command. At first, it may take more time to look up how to use these shortcuts, but eventually, with practice and drilling will become second nature and become real time savers.
Shell shortcuts
Navigating the Bash shell is easy to do. But it takes time to learn how to do well. Below are a number of shortcuts that make the navigation process much more efficient. I use nearly all of the shortcuts daily (except Ctrl + t and Ctrl + xx, which I only recently discovered). In a similar vein, I wrote a separate post long ago about setting up CLI shortcuts on iterm that can further augment the capabilities of the CLI.
Ctrl + a => Return to the start of the command you’re typing
Ctrl + e => Go to the end of the command you’re typing
Ctrl + u => Cut everything before the cursor to a special clipboard
Ctrl + k => Cut everything after the cursor to a special clipboard
Ctrl + y => Paste from the special clipboard that Ctrl + u and Ctrl + k save their data to
Ctrl + t => Swap the two characters before the cursor (you can actually use this to transport a character from the left to the right, try it!)
Ctrl + w => Delete the word / argument left of the cursor
Ctrl + l => Clear the screen
Ctrl + _ => Undo previous key press
Ctrl + xx => Toggle between current position and the start of the line
There are some nice Alt key shortcuts in Linux as well. You can map the alt key in OSX pretty easily to unlock these shortcuts.
Alt + l => Uncapitalize the next word that the cursor is under (If the cursor is in the middle of the the word it will capitalize the last half of the word).
Alt + u => Capitalize the word that the cursor is under
Alt + t => Swap words or arguments that the cursor is under with the previous
Alt + . => Paste the last word of the previous command
Alt + b => Move backward one word
Alt + f => Move forward one word
Alt + r => Undo any changes that have been done to the current command
Argument tricks
Argument tricks can help to grow the navigation capabilities that Bash shortcuts provide and can even further speed up your effectiveness in the terminal. Below is a list of special arguments that can be passed to any command that can be expanded into various commands.
Repeating
!! => Repeat the previous (full) command
!foo => Repeat the most recent command that starts with ‘foo‘ (e.g. !ls)
!^ => Repeat the first argument of the previous command
!$ => Repeat the last argument of the previous command
!* => Repeat all arguments of last command
!:<number> => Repeat a specifically positioned argument
!:1-2 => Repeat a range of arguments
Printing
!$:p => Print out the word that !$ would substitute
!*:p => Print out the previous command except for the last word
!foo:p =>Print out the command that !foo would run
Special parameters
When writing scripts , there are a number of special parameters you can feed into the shell. This can be convenient for doing lots of different things in scripts. Part of the fun of writing scripts and automating things is discovering creative ways to fit together the various pieces of the puzzle in elegant ways. The “special” parameters listed below can be seen as pieces of the puzzle, and can be very powerful building blocks in your scripts.
$* => Expand parameters. Expands to a single word for each parameter separated by IFS delimeter – think spaces
$@ => Expand parameters. Each parameter expand to a separate word, enclosed by “” – think arrays
$# => Expand the number of parameters of a command
$? => Expand the exit status of the previous command
$$ => Expand the pid of the shell
$! => Expand the pid of the most recent command
$0 => Expand the name of the shell or script
$_ => Expand the last previous argument
Conclusion
There are some many crevices and cracks of Bash to explore, I keep finding new and interesting things about Bash that lead down new paths and help my skills grow. I hope some of these tricks give you some ideas that can help and improve your own Bash style and workflows in the future.
, which I highly recommend.
