Setup of the evaluation

Our evaluation setup is based on the findings in the book Designing for interaction: Creating smart applications and clever devices by Dan Saffer. According to our interpretation of the book the following research and prototyping steps should be taken:

  • at least one Hunting statement is to be formulated,
  • On that basis a Prototype should be created,
  • A research survey is to be set up,
  • The survey is taken by our target group,
  • The results should be processed and conclusions drawn.


Research questions

Research questions are to be formulated as hunt statements. A hunt statement is a good tool to narrow down what to research and why. The more specific a hunt statement is, the better (how better it narrows down).

Our hunt statements should be about the adequacy of our information, both textual as well as visual. Does our website inform the visitor enough so that he understands the meaning of the project and the actions he should take? Is the website clear enough so that the user is able to navigate through the most basic steps to find information?

The following two hunt statements can be formulated:

  • We are going to research the information on our website so we can improve the understanding of our project.
  • We are going to research the basis of our website lay-out so that we can improve the understanding of our website and thus of our project.


The study

The shape of the study was that of a survey taken orally supported by an investigator. The subject took a survey with open questions on which, if necessary, the researcher provided additional information. This was especially the case with the part of the situation sketch. The subject had the opportunity to express its views and responses to the researcher. Subsequently the subject was also asked about his or her opninion, and there was room for discussion about the project, the survey and its content. Especially during this a lot of useful information came up to consider. Of course also some errors and ambiguities in the survey came up. All in all, survey lasted 30-45 minutes.
Open questions were chosen because of the nature of what we wanted to find out. The open questions and their answers, comments and discussions were the most useful tool to get information about effectiveness of the information flow.

The number of subject got stuck at 3 due to time constraints. However, they were varied 3: a man of 55 + (person 1) and one female student (21 – person 2) both not from Delft. The third person was a male student from Delft (23 – person 3). What was interesting that the non-Delft persons made comparisons with similar situations and projects in other cities (Maastricht, Utrecht).


  • At certain point individuals 1 and 2 became uncertain about the words ‘active’ and ‘resolved’ (‘actief’ and ‘opgelost’) on the ‘rabbit page’. First they thought that a rabbit appeared when active uneasiness would and once the uneasiness was solved the rabbit would disappear. initially it was not clear when rabbits are active and when they would disappear.
  • The information is missing that rabbits are actively placed in the streets.
  • Person 2 and 3 were wondering how much time sat between the rabbit placing and its disappearance (make way for work).
  • It was noted that you can upload pictures, but could they also be downloaded? (no download button).
  • The title ‘train rabbit’ (‘treinkonijn’) strongly suggested it only involved train travellers according to individual 1.
  • All individuals understood clearly that when they took part in the online discussion they were thinking and talking about the uneasiness and their solutions.
  • During the whole survey individual 1 continued to doubt about whether he did understand exactly what he should do during the project.
  • Both persons 1, 2 and 3 thought that the rabbit is the only form of communication of the inconvenience of the building activities to the outside world. 3 person noted that it would be annoying if you had to look for rabbits in this situation. Person 1 missed an overview in this situation.The map was interpreted by individual 1 and 2 as the place where the nuisance is resolved.

Final conclusion: from the different conclusions of the study can be concluded that the information about how the project functions is inadequate. What is striking is that links in the project are certainly understood, but the overview is missing. This has the consequence that the significance of links may be distorted. For example where person 2 first realizes that rabbits represent future changes, but this alters after seeing the rabbit page. The person then concludes that the rabbits represent current construction and that they will disappear when they are “resolved”. A structured and clear explanation of the project is desirable, at least improvement of the current information flow.

Conclusions about the survey

  • the use of the word passengers (reizigers) in combination with in the images of the brochures and poster gave the impression the project is only about train travellers, accorind to person 1.
  • Person 1 and 2 were both strongly inclined to comment on the functionality of the posters and flyers after seeing the pictures of it.
  • The paper survey is not the proper means to test the functionality of the layout of the website. Only person 3 had almost all the buttons and interactive features correct. The comments about not being able to intuitively explore the site indicate that there is a lack of an interactive prototype.

0 Responses to “Design evaluation”

  1. Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

May 2017
« Jan    

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 3 other followers

%d bloggers like this: