Welcome to STEAM
The Simple Team Experience Assessment Measure (STEAM) allows faculty to offer peer evaluations to student teams. A peer evaluation simply asks students how their peers performed on a team project. Peer evaluations, when properly implemented, allow an instructor both to assess how well individuals performed on a team as well as provide feedback to students so they become better at working on teams. For those unfamiliar with teams or peer evaluations a list of resources will be found later.
About STEAM
STEAM is a web-based peer evaluation tool that requires no software except an web-browser and access to the internet. STEAM is easy to use and can be customized for use an many different types of courses. To use STEAM:
- The instructor goes through the STEAM tutorial to set up the system or modifies a pre-existing evaluations.
- The instructor uploads a student roster and resets the system to make STEAM active.
- Students log in through a web browser to enter feedback about their team members. You can complete an example peer evaluation as a student here.
- Once all students on a team have completed the peer evaluation, the instructor reviews and approves evaluations.
- If the instructor wishes, students may receive individualized feedback about their performance on the team.
- At the conclusion of an evaluation data can easily be copied into a spreadsheet to record grades or perform more detailed analysis.
It will take less than one hour to learn to use the system and to configure a peer evaluation to give your students. Once you have an evaluation set up obtaining results takes very little time.
STEAM is highly configurable by the instructor to meet specific needs of their class. Five different evaluation tools can be fully configured by the instructor and almost any combination of tools may be used. Since STEAM is fully configurable to meet the needs of any class it is important to ensure the peer evaluation actually measures the intended team interactions and experiences. Those unfamiliar with setting up a peer evaluation should read the section below which provides resources to learn more about peer evaluations.
About Peer Evaluations
As with any measurement in education the results of a peer evaluation are a proxy, or indirect, measure of how students perform on a team. While proxy measures are never 100% accurate, a valid measure will provide accurate data on team dynamics that can’t be directly measured. Since an instructor has complete freedom to configure a STEAM peer evaluation, some thought needs to be given to what is being measured and how to measure team dynamics accurately. For those users who need some guidance or wish to learn more about peer evaluations, several references to educational literature are given below along with a brief summary. This list is not exhaustive and reflects the fact that STEAM was developed by engineering faculty.- B Knight, L. D. Fink, and L. K. Michaelsen, Team-Based Learning: A Transformative Use of Small Groups in College Teaching, published by Stylus Press, Sterling, VA, 2004. This book provides an overview of using teams in many different disciplines in higher education and is a good resource for most faculty.
- K. A. Smith, S. D. Sheppard, D. W. Johnson, and R. T. Johnson, Pedagogies of Engagement: Classroom-Based Practices, Journal of Engineering Education, vol. 94, p. 88, 2005. This paper is an excellent summary of data on the effectiveness of cooperative learning and how to effectively implement cooperative teams. If you are new to using teams this paper is highly recommended.
- CatMe is another peer evaluation tool based at Purdue that is available free to university faculty. While the CatMe system can’t be configured by the user the way STEAM can, it uses well validated questions and performs more in-depth data analysis than STEAM. Those who wish to use a simple, pre-configured validated peer evaluation should definitely consider CatMe.
- D.B. Kaufman, R. M. Felder, and H. Fuller, Accounting for Individual Effort in Cooperative Learning Teams, Journal of Engineering Education, vol. 89, pp.133-140, 2000. This paper discusses using a peer evaluation in a sophomore engineering course. The paper concludes that the peer evaluation given generally accurately measured students’ performance and many common concerns about peer evaluations were not observed.
- M. W. Ohland, R. A. Layton, M. L. Loughry, and A. G. Yuhasz, Effects of Behavior Anchors on Peer Evaluation Reliability, Journal of Engineering Education, vol. 94, pp.319-326, 2005. This paper discusses how giving students examples of behaviors associated with different ratings can improve the validity of peer evaluations. An exhaustive review of the peer evaluation literature is also included.
Pre-Configured STEAM Files
The STEAM system allows an evaluation to be saved to a file once it has been configured, then loaded at a later time. For users’ convenience several pre-configured evaluations are available that can be used as-is or modified. To use a pre-configured evaluation right-click the evaluation you wish to use and save the .txt file to your desktop. In STEAM go to Mange Evaluation Settings -> Load Settings and follow the instructions- Simple Team-Based Learning Evaluation: This single page evaluation has students report on the overall percentage of work performed by their peers.
- Seven Point Likert Evaluation: This simple form asks peers to rate each other on participation. The form mimics that of the Kaufman, Felder, and Fuller paper reference above.
- Team Project Evaluation: This evaluation uses four modules to evaluate student performance on engineering capstone design teams and is intended for in-depth evaluation of large team projects. Results of using this evaluation can be found here.
- Simple Salary Evaluation: Similar to the team-based learning evaluation above, but students are asked to provide a salary to each team member rather than a percentage rating. Often ratings that mimic grades are mistaken by students for grades.
Obtaining STEAM
STEAM is open-source and available at no charge provided users meet the terms of the Creative Commons license found at the bottom of all STEAM pages. If you would like to use STEAM two options are available. We will set up the system for you on our server or send you the software so you can install it on your own ASP enabled server. To use STEAM please send e-mail to kridnix@okstate.edu with a contact phone number. Once we verify you are a university faculty member we will send you the STEAM files or set up STEAM on our server, then give you an access password.
Accessing STEAM
Once STEAM is set up in a web-enabled directory instructors access STEAM via a web browser by navigating to http://site.name/path/proflogin.asp and logging in with the password you received when you contacted us. Here “site.name” is the name of the web site, and “path” is the directory on the web site where the evaluation is installed. We recommend you reset the default password at first login.
Students access the STEAM site using the team number and ID number entered on the roster you upload at http://site.name/path/studentlogin.asp
Questions
Any questions or bug reports should be sent to kridnix@okstate.edu.
