Evaluation of the design

Setup of the evaluation

Our evaluation setup is based on the findings in the book Designing for interaction: Creating smart applications and clever devices by Dan Saffer. According to our interpretation of the book the following research and prototyping steps should be taken:

  • at least one Hunt statement is to be formulated,
  • On that basis a Prototype should be created,
  • A research survey is to be set up,
  • The survey is taken by our target group,
  • The results should be processed and conclusions drawn.

Research questions

Research questions are to be formulated as hunt statements. A hunt statement is a good tool to narrow down what to research and why. The more specific a hunt statement is, the better (how better it narrows down).

Our hunt statements should be about the adequacy of our information, both textual as well as visual. Does our website inform the visitor enough so that he understands the meaning of the project and the actions he should take? Is the website clear enough so that the user is able to navigate through the most basic steps to find information?

The following two hunt statements can be formulated:

  • We are going to research the information on our website so we can improve the understanding of our project.
  • We are going to research the basis of our website lay-out so that we can improve the understanding of our website and thus of our project.

The study

The shape of the study was that of a survey taken orally supported by an investigator. The subject took a survey with open questions on which, if necessary, the researcher provided additional information. This was especially the case with the part of the situation sketch. The subject had the opportunity to express its views and responses to the researcher. Subsequently the subject was also asked about his or her opninion, and there was room for discussion about the project, the survey and its content. Especially during this  a lot of useful information came up to consider. Of course also some errors and ambiguities in the survey came up. All in all, survey lasted 30-45 minutes.
Open questions were chosen because of the nature of what we wanted to find out. The open questions and their answers, comments and discussions were the most useful tool to get information about effectiveness of the information flow.

The number of subject got stuck at 3 due to time constraints. However, they were varied 3: a man of 55 + (person 1) and one female student (21 – person 2) both not from Delft. The third person was a male student from Delft (23 – person 3). What was interesting that the non-Delft persons made comparisons with similar situations and projects in other cities (Maastricht, Utrecht).


  • At certain point  individuals 1 and 2  became uncertain about the words ‘active’ and ‘resolved’  (‘actief’ and ‘opgelost’) on the ‘rabbit page’. First they thought that  a rabbit appeared when active uneasiness would and once the uneasiness was solved the rabbit would disappear.  initially it was not clear when rabbits are active and when they would disappear.
  • The information is missing that rabbits are actively placed in the streets.
  • Person 2 and 3 were wondering how much time sat between the rabbit placing and its disappearance (make way for work).
  • It was noted that you can upload pictures, but could they also be downloaded? (no download button).
  • The title ‘train rabbit’ (‘treinkonijn’) strongly suggested it only involved train travellers according to individual 1.
  • All individuals understood  clearly that when they took part in the online discussion they were thinking and talking about the uneasiness and their solutions.
  • During the whole survey individual 1 continued to doubt about whether he did understand exactly what he should do during the project.
  • Both persons 1, 2 and 3 thought that the rabbit is the only form of communication of the inconvenience of the building activities to the outside world. 3 person noted that it would be annoying if you had to look for rabbits in this situation. Person 1 missed an overview in this situation.The map was interpreted by individual 1 and 2 as the place where the nuisance is resolved.

Final conclusion: from the different conclusions of the study can be concluded that the information about how the project functions is inadequate. What is striking is that links in the project are certainly understood, but the overview is missing. This has the consequence that the significance of links may be distorted. For example where  person 2 first realizes that rabbits represent future changes, but this alters after seeing the rabbit page. The person then concludes that the rabbits represent current construction and that they will disappear when they are “resolved”. A structured and clear explanation of the project is desirable, at least improvement of the current information flow

Conclusions about the survey

  • the use of the word passengers  (reizigers) in combination with in the images of the brochures and poster gave the impression the project is only about train travellers, accorind to person 1.
  • Person 1 and 2 were both strongly inclined to comment on the functionality of the posters and flyers after seeing the pictures of it.
  • The paper survey is not the proper means to test the functionality of the layout of the website. Only  person 3 had almost all the buttons and interactive features correct. The comments about not being able to intuitively explore the site indicate that there is a lack of an interactive prototype.


as concluded the information about the project should be improved. We’ve done the following suggestions to improve the website, our main information source. We’ve taken the conclusions of the study into account.

  • Showing a modified version of the movie presentation. When the website is visited (for the first time) the movie will be shown in advance of the homepage. This movie is the same as the presentation movie (see link), only the spoken text will be adjusted to a specific audience, namely the participants. The function of the film is to inform visitors about the various links in and the progress of the project. The video presentation has pretty much the same purpose and is therefore suitable, in a modified form, for this purpose.
  • Inserting a descriptive text for each page, with the focus on the present issues. At the moment there are hardly any descriptive texts about the content on the various web pages. It is assumed that the  content is clear. You usually can click on the headings to get information about the header and the corresponding section. If we give the information directly (in abbreviated form) the comprehensibility of the subjects improved. There will be no uncertainty about entitlement such as the project name ‘train rabbit’ ‘and heads’ active’ (‘actief’) and ‘dissolved’ (‘opgelost’) rabbits. It is important to notice that the layout of the site is not affected if this is applied. You could solve this by a popup to display information. The new site recognizes a user and then only popup is displayed when a new user visits the site. If this popup is clicked away, it will disappear into the corresponding button, so users know where you can retrieve this information again.
  • Giving an overview page. Making a page fully dedicated to describing the project, which is linked  to the homepage in the same form as the home page and overview rabbits. This way, all information is gathered and easy to find.
  • Some facts about the project are currently not known. These can be detected by an extensive analysis of the information from the website. What is already known to be unclear:
  1. the duration of the project,
  2. the time between the placement of a rabbit and the disappearance (and thus the start of construction),
  3. the absence of the knowledge that the rabbits are placed actively by project ‘Treinkonijn’,
  4. that ‘Delft Bouwt’ is the official channel of information about the construction and that our project is not the major and only information about it.

These facts can be added to the given information in one of the above suggestions.


0 Responses to “Evaluation of the design”

  1. Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

January 2010
« Dec    

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 3 other followers

%d bloggers like this: