I've deployed many usability tools throughout my career and generally when I find one that I like I stick with it. I also have discovered that a tool is only as good as the person using it. This week I had the opportunity to try out Loop11. The website says, "Loop11 is an application that lets you run your own usability studies without the need for a usability lab, specialized equipment and a moderator. Simply determine the tasks you want to test on your website, recruit some participants and launch your study."
I selected a website, created tasks and set up survey questions for participants to evaluate it using Loop11. My goal was to discover how simple hipmunk is to book a flight and vacation destination. I walked-through previewing the test on Loop11 multiple times imagining that I was the user taking it. Each time I refined the tasks and survey questions to make them as clear as possible. I did not want to lead the user and I did not want to have them wander aimlessly at the same time.
Overall, I liked the simplicity of Loop11 - it's easy to take and set up the test. For the user, during the test they mark a task "Complete Task" or "Abandon Task". Then the participant answers survey questions after each task as followup. For the UX professional, I like the fact that the usability tool captures screenshots of user throughout the process, measures completion rate of the task, can easily filter the results and gives many options to deploy it.
The one technical issue that came up during the test seemed to be a limitation between the Loop11 software and website that I picked for this test. For my assignment, I generated a Loop11 link to the session that I emailed to my participants. 6 out of 10 users were able to complete the pre-survey questions and complete the test successfully. However 4 out of the 10 users only got past the pre-survey questions but were unable to load the site within the Loop11 software because the browser had a bad request. Next round, I will QA test the site during final review with a tool like Browserstack to insure it's technically sound.