Speaking Event – The Science of Diversity

Myself and two of my colleagues will be talking about the science of diversity and whether diverse teams are better than non diverse teams….
If you are in London please come along! http://www.thoughtworks.com/events/science-diversity
——————————————————
ThoughtWorks would like to invite you to meet with us at an upcoming event. Please share!
 
ThoughtWorks deliver software to people with ambitious missions and we’re looking for talented techies to join us. We value diversity and on 18th November we are hosting an evening with our CTO Rebecca Parsons who will be taking a scientific and technical approach to answer the question ‘are diverse teams better?’ 
 
There will also be an opportunity to network and find out about careers with ThoughtWorks. 
You can register here – we look forward to meeting you!
Please contact Katie kpartridge@thoughtworks.com @katielisa52 for more details.Inline images 1
Advertisements

Creating C# .Net Websites in IIS with a Powershell Script

I created this little script to automate the creation of websites in IIS. It’s written in powershell.

You will need to run this script in powershell thats running in Administrator Mode.

Prerequisites:
You will need to have .net installed.

Step 1 Loading the Web Administration Module

This script is using the WebAdministration module to do the creation and removal of websites. To be able to load this module before it runs my powershell script I create a .cmd file that I would call that would load this module and then call my main powershell script.

install_website.cmd

powershell -executionpolicy remoteSigned -command import-module .\PSCommon\WebAdministration;.\install_website.ps1

Step 2 Creating the main .ps1 script

Create a new script, I called mine install_website.ps1. I put mine next to my install_website.cmd.

install_website.ps1

C:\Windows\Microsoft.NET\Framework\v4.0.30319\aspnet_regiis.exe -i

$website_name = "MyWebsite"
$website_source_directory = resolve-path "..\MyWebsite"
$website_virtual_directory = "IIS:\Sites\MyWebsite"

$app_pool_name = "MyWebsite_AppPool"
$app_pool_virtual_directory = "IIS:\AppPools\MyWebsite_AppPool"

$sub_application_name = "SubApplication"
$sub_application_source_directory = resolve-path "..\MyWebsite.SubApplication"

Function CreateWebsite {
 New-WebAppPool $app_pool_name
 Set-ItemProperty $app_pool_virtual_directory managedRuntimeVersion v4.0
 New-Website -Name "$website_name" -PhysicalPath $website_source_directory -ApplicationPool "$app_pool_name" -Port 8082 -Force
 New-WebApplication -Name "$sub_application_name" -Site "$website_name" -PhysicalPath $sub_application_source_directory -ApplicationPool $app_pool_name
}

Function RemoveWebsite($app_pool_virtual_directory, $site_virtual_directory, $website_name){
 if (Test-Path $app_pool_virtual_directory){
   $webSite = Get-Item $site_virtual_directory
 if ((Get-WebsiteState -Name "$website_name").Value -ne "Stopped") {
   $webSite.Stop()
 }
 $poolName = $webSite.applicationPool
 $pool = Get-Item "IIS:\AppPools\$poolName"
 if ((Get-WebAppPoolState -Name $poolName).Value -ne "Stopped") {
   $pool.Stop()
 }
 Remove-Website -Name "$website_name"
 Remove-WebAppPool -Name $poolName
 }
}

RemoveWebsite $app_pool_virtual_directory $website_virtual_directory $website_name
CreateWebsite

As you can see I have two functions. One creates my website and associated app pool, the other removes it if it already exists. So I can run my script multiple times without a problem.

CreateWebsite, first sets up a new app pool, sets its ManagedRuntimeVersion to v4.0, creates a new website and then we also needed a sub application below that, so also sets that up.

Automating WebPageTest with a Ruby Script

WebPageTest is a great tool for seeing how well your website is performing. I was just on a project where the speed of a particular results page loading was the focus. How long the server took to respond, how long the UI took to render, what it was rendering, and where there were places for optimisations. We also wanted to measure the users’ perceived page load time.
We wanted to create an automated test that would run regularly so that we could track changes in page load over time.

Reasons why we picked WebPageTest

  • Ability to test any URL we wanted (ourselves, competitors, etc.)
  • Multiple browsers that can be used to test pages
  • Multiple locations and line speeds to test from
  • Can be automated (they have a REST API)
  • Ability to pin point a particular DOM element to measure load time of. For example we might consider the user perception of the page load as when there are 10 results on the page.
  • Provides a waterfall of all the objects loaded onto the page so that we can look for optimisations.

We set up the following to prototype (read: not production quality)  a top line measurement tool that the business could use to see how performance was tracking from a users perspective. With only a week of these tests running we found 3 issues that had been causing major problems with the page load time.

Where to start?

WebPageTest provide a RESTful API that we can use to interact with them. You will however have to host your own  private instance of WPT to be able to automate public URLs. You will also have to set up an agent for that instance to talk to.

Follow this guide for more information Setting up a Private Instance of Web Page Test. We set up  a couple of Amazon EC2 boxes for ours. (Make sure you pick at least Small rather than Micro)

Once you have that setup you should have a private instance of WebPageTest and URL that you can use to start manually testing your website.

For the purposes of this example I am going to test the BBC news website… I also used Ruby for this script. This was my first ever time creating a Ruby script for something like this, so keep that in mind!

These are some steps you can follow to build up a scripted test, and then store the results in a MongoDB for analysis later.

Step 1:

Call your instance of WebPageTest with your URL of choice and get back a 200 response

#!/usr/bin/env ruby
require "net/http"
require "uri"

test_url = "http://www.bbc.co.uk/news/"
baseurl = "http://[WEB PAGE TEST SERVER]/runtest.php?runs=1&f=xml&fvonly=1&url=#{test_url}"

response = Net::HTTP.get(URI(baseurl))
puts response

At this point you should get a response that looks something like this:

<!--?xml version="1.0" encoding="UTF-8"?-->

  200
  Ok
  <data>
    130723_9Y_BM
    d93f0a316369c61a891428dfb8b071d97b3dd19b
    http://[WEB PAGE TEST SERVER]/xmlResult/130723_9Y_BM/
    http://[WEB PAGE TEST SERVER]/result/130723_9Y_BM/
    http://[WEB PAGE TEST SERVER]/result/130723_9Y_BM/page_data.csv
    http://[WEB PAGE TEST SERVER]/result/130723_9Y_BM/requests.csv
    http://[WEB PAGE TEST SERVER]/jsonResult.php?test=130723_9Y_BM/
  </data>

WebPageTest will asynchronously run the tests and dump the results into the locations returned above. As you can see there are lots of different formats that you can consume for the results data. Everything that I needed was in the summaryCSV.

Step 2:

Parse the URL to get the summaryCSV and then poll that until the results appear (as we have no idea of knowing when they will appear I checked every 5 seconds until I no longer got a 404 response). I then parse the CSV response into a 2 dimensional array, where the first array are the headers, and the second is their values. You can then see all the results that come back and which ones might be interesting for you.

#!/usr/bin/env ruby

require "net/http"
require "uri"
require "nokogiri"
require "csv"

test_url = "http://www.bbc.co.uk/news/"
baseurl = "http://[WEB PAGE TEST SERVER]/runtest.php?runs=1&f=xml&fvonly=1&url=#{test_url}"

response = Net::HTTP.get(URI(baseurl))
doc  = Nokogiri::XML(response)

csv_url = doc.at_xpath('//summaryCSV').content

uri = URI.parse(csv_url)
http = Net::HTTP.new(uri.host, uri.port)
req = Net::HTTP::Post.new(uri.path)

while((csv_content = http.request(req)).class == Net::HTTPNotFound)
	sleep 5
end

raw_results = CSV.parse(csv_content.body, {:headers => true, :return_headers => true, :header_converters => :symbol, :converters => :all})

puts raw_results

Step 3:

Great! Now I have test results coming back and I want to save the ones I’m interested in so that I can visually represent them later and look at the changes in performance over time.
I’m going to use mongoDB to store the results of the tests. I created a class called TestResult with the fields that I am interested in from my WebPageTest results.
At this point you will need mongoDB up and running and you will also need a .yml file to define your mongoDB setup.
mongoid.yml

development:
  sessions:
    default:
      database: web_page_test_results
      hosts:
        - localhost:27017

My scripts now looks like this:

#!/usr/bin/env ruby

require "net/http"
require "uri"
require "nokogiri"
require "csv"
require "mongoid"
require "chronic"

Mongoid.load!("mongoid.yml", :development)

class TestResults
	include Mongoid::Document

	field :timestamp_of_test, type: Time
	field :load_time, type: Integer
	field :time_to_first_byte, type: Integer
	field :csv_url, type: String
end

test_url = "http://www.bbc.co.uk/news/"
baseurl = "http://[WEB PAGE TEST SERVER]/runtest.php?runs=1&f=xml&fvonly=1&url=#{test_url}"

response = Net::HTTP.get(URI(baseurl))
doc  = Nokogiri::XML(response)

csv_url = doc.at_xpath('//summaryCSV').content

uri = URI.parse(csv_url)
http = Net::HTTP.new(uri.host, uri.port)
req = Net::HTTP::Post.new(uri.path)

while((csv_content = http.request(req)).class == Net::HTTPNotFound)
	sleep 5
end

raw_results = CSV.parse(csv_content.body, {:headers => true, :return_headers => true, :header_converters => :symbol, :converters => :all})

TestResults.create({
	:timestamp_of_test => Chronic.parse(raw_results[1][:time]),
	:load_time => raw_results[1][:load_time_ms],
	:time_to_first_byte => raw_results[1][:time_to_first_byte_ms],
	:csv_url => csv_url
	})

And my mongoDB happily contains the result of my first test, which will look something like this

db.test_results.find()
{ "_id" : ObjectId("51eef7cee055e6cee5000001"), "timestamp_of_test" : ISODate("2013-07-24T21:38:04Z"), "load_time" : 3566, "time_to_first_byte" : 317, "csv_url" : "http://[WEB PAGE TEST SERVER]/result/130723_RW_CN/page_data.csv" }

What next?

If I had had a chance to extend this, I would of loved to of added visualisation, using a graphing framework and deploying the results to a web service so that everyone can see the change over time and drill down into any of the suspicious looking results.

The fear of blogging about technical topics…

I rarely blog about technical topics, despite my main trade of being a developer. The prospect has always scared me, and from others that I’ve talked to I’m not the only one.

The irrational thoughts that tend to go through my head are:

  • What right to I have to blog about technical stuff?!
  • Major case of Imposter Syndrome
  • What do I know that the internet doesn’t already?!
  • Will people judge me about what knowledge I do have?
  • What if  I’m wrong?!

Wonder if anyone else finds this too?

Well, we have to embrace whats scary to grow. So I have set myself a target of at least one technical blog a month.

These are my mantras to help me feel more confident about it

  • We are all always learning. There will always be people that know more about something than you, but there will always be people who know less. Things that I find useful when I am learning WILL be useful for other people who are learning.
  • If I had to hunt about on the internet for half an hour to find out how to do something,  if I can give an example that might save someone  haf an hour of googling, then I have made a tiny difference.
  • I must embrace feedback! If people leave comments, and share other ways that I could do things better, I should embrace and learn from these.
  • I am not ever pretending to know everything… these are just things that have helped me and I hope they might help others.
  • I can put things down here, so I can remember them for next time.

Here goes!

I’d love to hear any thoughts if you feel the same about technical blogging, or even better if you used to, and how you got over it.

The Retrospective Handbook by Patrick Kua

The Retrospective Handbook

One of my colleagues at ThoughtWorks published his first book last year called the Retrospective Handbook. It’s a great tool for people thinking about facilitating or wanting to hone their skills facilitating Agile retrospectives.

You can buy it here on Amazon or on Leanpub

I thought it was great… I’ve listed out here some of the things that the book covers that I really liked.

  • The right context for Retrospectives
    • The effect that team dynamics and environment have on the success of a retrospective
  • Lots of practical advice
    • How to prepare and set up a retrospective. Right down to the kinds of pens and paper best to use.
    • Lots of examples of language to use in specific situations
  • The importance of independent facilitators and advice on what to do if thats not possible and a team member facilitates.
  • Great tips for facilitators
    • How to help achieve equal participation from everyone.
    • How to handle anti-patterns from participants
    • Importance of observing body language as a facilitator and realising that there are many more things to ‘listen to’ of you participants that just what they are speaking out loud.
  • A whole section on Distributed Retrospectives
  • Your job is not done once people have left the room. There are a bunch of necessary things that need to get done after the retro is over.
  • 8 Common retrospective smells

Stand-ups – We can make them better…

I’ve been thinking about how to get the most out of stand-ups. They are something that all the teams that I have been on do every day, but  how often do they actually meet the real goals of a stand-up? I spent some time thinking about how, instead of just rattling off the usual status update, we can really think about how to best use this time. Not only to communicate  with and learn from the team, but also how to provide that daily re focus on what the team needs to do today to get awesome features into the hands of users. I propose some different questions to think about and frame your updates with during stand-up.

While I was hunting about the internet looking for inspiration, I came across Jason Yipp’s article on Martin Fowler’s Blog – It’s Not just Standing Up. There’s loads of great stuff in there, but specifically what jumped at me was the goals of a stand-up.

 The Goals of the Daily Stand-up are GIFTS

There are several goals for a daily stand-up  meeting:

  • To help start the day well
  • To support improvement
  • To reinforce focus on the right things
  • To reinforce the sense of team
  • To communicate what is going on

As a mnemonic device think of GIFTS:

Good Start, Improvement, Focus, Team, Status

I also came across this from Agile Coaching by Rachel Davie and Liz Sedley. A good way to sanity check how your stand-ups are going.

Signs of a healthy stand-up

– Is everyone engaged, motivated and excited?

– Are they making progress and working on high-priority tasks?

– Are they working together and helping each other?

– Are they able to concentrate and do their job without disruptions?

Reality of what many stand-ups tend to be and some of the problems

I think that it’s easy to say that you are ‘doing standups’, everyone goes around giving their 3 question updates (what I did yesterday, what I am planning on doing today and do I have any blockers), but  you have to ask how effective they are. Was everyone listening to each other? Do you even remember what each person said? Did you learn anything from stand-up? Do you have a better sense of where in the lifecycle any particular story is?

Just like other processes that the team follows, stand-ups should be continually improved and tweaked.

So. I don’t think that the 3 questions above are enough. I think they are great to get the team started, but teams often grow out of them quickly. As long as you keep the goals in mind, you can change the structure to be whatever you want it to be.

Why aren’t they enough? Some reasons:

  • There is too much talk of ‘I’.
  • Lacks focus on getting stories to ‘Done’.  Ultimately the team should be focused on how to get the best, most useful features into the hands of it’s customers, I would like to see updates the revolve around how to achieve that.
  • Too easy for people to hide behind them, robotically answering and it’s easy to lose emphasis.
  • The language is quite passive. Rather than stating what your blockers are, why not focus on what you are doing to get rid of them?

The differences may seem subtle, but the language a team use can totally change the team’s approach.

How to make them better?

Don’t think of it as a status report, more as a chance to communicate with and update your whole team at once.

Explore different formats of standups. I find that ‘Walk the Wall’ works quite well to ensure that everyone is focused on what needs to get done to get  features out the door.

Start to move away from the 3 basic questions. Talk with your team about what kind of updates you would like to hear during stand-up that would be most useful to everyone.  Change the language to be more proactive.

Perhaps think about framing your updates using the language below:

  • What am I doing today to get cards moved across the wall towards “Done”?
    • Ensures that the focus is on activities that directly relate to helping to progress work.
  • What remaining work is left? When am I likely to move it to the next state?
    • Helps downstream roles to know when to expect new work, helping them plan better. Gives the whole team the sense of where the story is in its lifecycle.
  • Meetings? Would the team benefit from hearing a summary?
    • I forever hear “I was in meetings all day”,  hopefully the meetings were project related, so share what happened. People want to know…
  • Did you learn something yesterday that people might benefit from knowing?
    • Help the team become more efficient, by sharing things that you have learnt. It may prevent someone from making a similar mistake, or struggling with something someone has already figured out. If it prompts interest, have a post-standup huddle for people to hear more.
  • What am I doing to remove/fix blockers?
    • This is an important language difference from the standard update.  Don’t wait for people to remove obstacles for you, see what you can do yourself to work round them or move them yourself, or even other peoples. It’s more often than not quicker than waiting for someone else to do it.
 So I prepared the following ‘cheat sheet’ to help you think about your update. You could even stick this up on the story wall itself to act as a memory trigger.

StandupOther great stand-up resources

It’s Not Just Standing Up: Patterns for Daily Standup Meetings http://www.martinfowler.com/articles/itsNotJustStandingUp.html

The Standup Game – Great game to introduce standup concepts, but also to bring awareness to the effects that stand up anti-patterns have on the team.  http://loveagile.com/stand-up/stand-up-game

Some ways to address anti-patterns http://www.scrumalliance.org/articles/358-daily-standup-beyond-mechanics-a-measure-of-selforganization

Gender Diversity on Teams – Roundup

Wanted to keep a note of a bunch of articles and books that I found recently around studies about gender diversity on teams. Prompted by a discussion with a programme manager at an investment bank and then subsequently with my fellow ThoughtWorks colleague Nic Ferrier

I found these often quoted “facts” and was trying to hunt down their sources…..

National Center for Women in Technology – Resources
The Impact of Gender Diversity on the Performance of Business Teams 
Gender Diversity and the Impact on Corporate Performance – Credit Suisse Research Institute
Gender Diversity, Team Decision Quality, Time on Task, and Interpersonal Cohesion
LBS study shows addition of women to teams improves performance