Fix Google Analytics search queries in WordPress

I embarrassingly discovered the other day that I was not receiving metrics or analytics about keyword queries in the Google Analytics console.  It turns out that problem was twofold.  First, I didn’t have the SSL version of my site enabled in the Google webmaster tools and second, I was serving a cached version of my sitemap that was several months out of date.

To give you an idea of how this issue manifested itself, and how I discovered that there were issues in the first place – here’s what my search keywords were showing as in the Google Analytics console.

search queries

Clearly the data is less than useful.  The solution to this problem is pretty easy to fix at least.

Fixing the webmaster properties

Open up the Google webmasters site (you should have this setup already, if not go ahead and get signed up and add your WordPress site).  If you have recently updated your site to use https, make sure you add a new property in the webmaster tools for the https version of your site that matches your http version.

Doing this will tell google to keep track of search queries for the https version of your site, which should be the default after swtiching.  After adding the new https property and indexing it, give it a day or two to start collecting metrics, and check back.  Now when you check your search query traffic in the webmaster tools you should start to see all of the search results.

search queries

Also be sure to update properties to use https in both the Webmaster console as well as the Analytics console.  For example, in the Analytics console under Admin -> Property settings -> default URL, there is an option to use http or https.  Likewise, in the webmaster console there is an option for defaulting to http, which is buried in the Google Analytics interface under Admin -> Property settings -> Search Console.  Make sure you update ALL of the site settings to use https.

NOTE: It can take some time for queries to begin showing up in the Google Analytics console (it took about two days for them to start showing up for my site after fixing all the https references).

Fixing sitemaps issues

If you find that Google isn’t indexing and using all of your posts and pages, the next thing to look at the sitemaps.  A quick way to know if you your sitemaps file is doing its job is to pull open the sitemaps, which can be found under the Crawl -> Sitemaps menu.

webmaster tools sitemap

The above shows what a healthy sitemap index looks like (after I corrected the problem).  There is a button located in the top left of this view that can help you test your sitemap while you are updating your settings.  First check for any items in the “issues” column.  Also, if the “processed” date here isn’t recent then there is probably an issue.  One last thing to check – if there are either no entries in this view or fewer then you expect, something is probably not working.

There are many more knobs and dials you can adjust in the webmaster tools, so if you haven’t played with them much I would recommend spending some time and poking around.

I should quickly mention that my solution assumes that you are using the Google XML Sitemaps plugin.  If you’re not using this plugin, and you are either 1) new to WordPress or 2) don’t want to manage your sitemaps file manually, I suggest you enable it.  It makes sitemap management so much easier.

After you have the plugin turned on, navigate to your blog settings for sitemaps, which can be located in Dashboard -> Settings -> XML-Sitemap.  Clicking the popup should bring up a page similar to the following.

xml sitemaps

First, make sure everything looks correct in the settings.  If you are setting this up for the first time you might need to configure some of the settings.  For example, make sure the site name matches the listing, and the options to notify search indexers are all turned on.

When I was troubleshooting the search queries not getting set, I navigated to this menu and immediately noticed that the plugin was showing a warning about using a cached version of my sitemaps.xml file.  To fix this warning, there should be an option to remove the cached versions.

Next, there should be an option near the top of the sitemap settings to “notify search engines about your sitemap”.  After you have adjusted all the sitemap settings and cleared the cached sitemaps file, click that link to trigger a ping to the search indexers.

Be aware that the crawling process may take up to a few days to index and update so be patient.

Read More

Defaulting Google search results to the past year

If you spend as much time looking around the internet for answers to obscure questions as I do, you quickly realize that often times, Google will happily present you with results that are many years old and out of date.  This is especially frustrating when you eventually find an answer after spending quite some time searching, only to realize the answer you’re looking for is 5 years old.  For more reading on efficient searching check out this post on the 10 tab rule, which has some useful ideas for better searching in general.

xkcd

There is a trick that will allow you to customize the Google default results.  The key to mapping the Google search terms is really just the URL.  Google uses search parameters for querying, so you can do some really cool things.  Obviously this is a powerful concept, so a lot of useful searches can be mapped.  This idea can be taken further to map keywords to other searches, like YouTube, Google maps, Stack Overflow, etc, or basically any site that provides a search interface.

I have only tested these key/search mappings for Google search results on Google Chrome, so if you use another browser there might be a similar trick, I just haven’t attempted it.  Open Chrome settings and navigate to the “Search” section.

This will pull up a dialog box with a list of default search engines.  Scroll to the bottom of the list and add the following values to the corresponding fields from the screenshot.

The Keyword is just a “>” symbol, and it can be anything really.  I chose the > because it is quick and easy to get to.  The rest is pretty self explanatory.  After the entry has been added, scroll through the search engines and find the new “Google recent” entry – there is a button that says “Make default” if you hover over the search engine entry.  Click that and then click done.

Now when you do a Google search from your search bar it will default to items from the past year.

Bonus

You can extend this trick further to map keys in your Google search bar to do other searches.  For example, you can map a key (or word) to search for image results.  In the below example, I am using “I” as the keyword.

After adding the above snippet into the search settings you can navigate to the search bar, type in I (followed by a tab) and the term to search for and it will automatically do an image search.  The key to making the mapping being a tab completion in the search bar is the q=%s part in the URL.

The last bonus search that has worked for me is the “feeling lucky” search.

That keyword (I used a tilde) can once again be anything, but preferably should be fast to get reach to make searching easier.

One final note

Sometimes you actually do want to search for results that are over a year old.  This is true of information that doesn’t really change often.  So if you are having trouble finding a website you think should be at the top of the search, make sure that the default search result is set to any time.  Ideally you would make another key mapping to handle this searching behavior.

Read More

awesome-rancher

Awesome lists are a newish phenomenon for organizing links and information that have been gaining popularity on Github.  You can read more about awesome lists here, but essentially they are curated lists of helpful links and other various resources that are put together by members of their communities.

There are lots of awesome lists these days on Github, and there are even awesome lists for non-tech categories, including recipes and video games. Oddly enough though through my own searching, I quickly discovered that there was no awesome list for Rancher.

So I decided to give a little bit back to the Rancher community in my own way.  If you aren’t aren’t familiar with Rancher, it is essentially a platform and orchestration engine for container based workloads.  Out of all of the container management and orchestration platforms that have been growing out of the container popularity explosion, using Rancher has definitely the most pleasant experience and easiest to use.  Also, if you are new to Rancher, please go check out the list on Github for more information, that is what it is there for.

Check out the awesome-rancher project here.

The goal for me and for the project is to make awesome-rancher the go to resource for finding useful information.  Therefore, I want awesome-rancher to be as easy to use and as complete as possible so new users have a good jumping off place when they decide to dive into Rancher.

If you are a Rancher community member or even just notice a key resource missing from the list, please either let me know with a comment here, on twitter/facebook, or just open a PR on the project itself (which is probably the easiest).

Read More

Curl on Windows using a Docker wrapper

Does the Windows built-in version of “curl” confuse or intimidate you?  Maybe you come from a Linux or Unix background, and yearn for some of your favorite go-to tools?  Newer versions of Powershell include a cmdlet for interacting with the web called Invoke-WebRequest, which is useful, but is not a great drop in replacement for those with experience in non Windows environments.  The Powershell cmdlets are a move in the right direction to unifying CLI experiences but there are still many folks that have become attached to curl over the years, including myself.  It is worth noting that a Windows compatible version of curl has existed for a long time, however it has always been a nuisance dealing with the zip file, just as using SSH has always been a hassle on Windows.  It has always been possible to use the *nix equivalent tools, it is just clunky.

I found a low effort solution for adding curl to my Windows CLI flow, that acts as a nice middle ground between learning Invoke-WebRequest and installing curl binaries directly, which I’d like to share.  This alias trick is a simple way to use curl for working with API’s and other various web testing in Windows environments without getting tangled in managing versions, and dealing with vulnerabilities.  Just download the latest Docker image to update curl to the newest version, and don’t worry about its implementation across different systems.

Prerequisites are light.  First, make sure to have the Docker for Windows app installed (stable or beta are both fine) as well as a semi-recent version of Powershell.

Next step.  If you haven’t set up a Powershell profile, there are also lots of links and resources about how to do it.   I even wrote about it recently, so I am skipping that step as well.  Start by adding the following snippet to your Powershell profile (by default located in C:\Users\<user>\Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1) and saving.

# Curl alias using docker
function Docker-Curl {
   docker run --rm byrnedo/alpine-curl $args
}

# Aliases
New-Alias dcurl Docker-Curl

Then source you terminal and run the curl command that was just created.

dcurl -h

One issue you might notice from the snippet above is that the Docker image is not an “official” image.  If this bothers you (security concerns, etc.), it is really easy to create your own, secure image.  There are lots of examples of how to create minimal images with Curl pre-installed.  Just be aware that your custom image will need to be maintained and occasionally rebuilt/published to guard against future vulnerabilities.  For brevity, I have skipped this process, but here’s an example of creating a custom image.

Optional

To update curl, just run the docker pull command.

docker pull apline-curl

Now you have the best of both worlds.  The built-in Invoke-WebRequest cmdlet provided by Powershell is available, as well as the venerable curl command.

My number one case for using curl in a container is that it has been in existence for such a long time (less bugs and edge cases) and it can be used for nearly any web related task.  It is also much handier to use curl for those with a background using *nix systems, rather than digging around in unfamiliar Powershell docs for similar functionality.  Having the ability to run some of my favorite tools in an easy, reproducible way on Windows has been a refreshing experience while sliding back into the Windows world.

Read More

Limit Jenkins Multibranch Pipeline Builds

As the Jenkins pipeline functionality continues to rapidly evolve – the project documentation (or lack thereof), has been a consistent pain point as a user. Invariably, the documentation is either out of date or completely missing.  I expect the docs to improve as the project matures, but for now, the cake is a lie.  I ran into this roadblock recently, looking for a way to limit the number of concurrent builds that happen in Jenkins, using the pipeline.  In all of my anguish, I hope this post will help others in avoiding the tediousness of finding the seemingly simple functionality of limiting concurrent builds, as well as give some insight into strategies for figuring out how to find undocumented features in Jenkins.

While this feature is fairly obvious for old-style Jenkins jobs, a simple check box in the job configuration – finding the same functionality for pipelines is seemingly non existent.  Through extensive Googling and Stack Overflowing, I discovered this feature was recently added to the Multibranch plugin.  Specifically, I found an issue in the (awful) issue tracker used by Jenkins, which in turn led me to uncover some code in a semi recent PR that basically allows concurrency to be turned on or off.  Of course when I tried to use the code from the PR it didn’t work right away.  So I had to go deeper.

Eventually, I  stumbled across a SO post that discusses how to use the properties functionality of pipelines.  Equipped with this new piece of information, I finally had enough substance to start playing around with the code.  To make the creation of pipelines easier, Jenkins also recently added a snippet generator, which allows users to build out sample snippets quickly.

To use the snippet generator, either drill into an existing pipeline style job using a similar URL as below:

https://jenkins.example.com/job/<jobname>/pipeline-syntax/

Or create a new job, and click on the “Pipeline Syntax” link after it has been created to test out different snippets.

pipeline syntax

Inside the snippet generator there are a number of “steps” to choose from.  From the information I had already gathered, I just selected the properties step to create the basic skeleton of what I wanted and was able to use the disableConcurrentBuilds() function I found earlier. Below is a snippet of what the code in your Jenkinsfile might actually look like:

node {
 // This oneliner is what limits concurrent builds
 properties([disableConcurrentBuilds()])

 // Do stuff
 ...
}

Yep.  That’s it.  Just make sure to put the properties() function at the beginning of the node block, otherwise concurrency won’t be adjusted right away and could lead to problems.  Another thing to note; the step to disable concurrency could just as easily be moved into workflow libraries and applied at the global level and applied at the beginning of all jobs if you wanted to limit concurrency for all pipeline builds, since the code is just Groovy.  Finally, the code will disable concurrent builds on a per branch basis.  Essentially, if you push many different branches it will still build all of them, it will just limit each branch to one build at a time and will queue up jobs for any commits that get pushed after the initial job has been created.  I know that is a mouthful.  Let me know in the comments if this explanation needs any clarification.

While I love open source software, sometimes project’s move so fast that certain areas of it get neglected.  I am thankful for things like Github, because I was able use it to piece together all the other information I found to come up with a solution.  But, I would argue having good documentation not only saves folks like me the time and energy of the crazy searches, it also makes it much easier for potentially new users to look at, and understand what is going on.  I will be 100% honest and say that Jenkins pipelines are not for the faint of heart, and I’m sure there are many others who will agree with this sentiment.  I know it is easier said than done, but anything right now would be an improvement in my opinion.

Read More