Innovations, Improvements, Research

FA 2013 Project Second iteration of the pilot with improvements: Pairing a college reading course with a content course and using the Effective Lifelong Learning Inventory with adaptive learning technology http://sites.psu.edu/fa13project/

FA 2012/SP 2013
Pairing a Reading Course with a Content Course and Tracking Student Success

During the Fall 2012 semester, I worked on a pilot project (special thanks to Mark Casteel (psychology), Kathleen Jansen (psychology), and Danielle Runkle (LL ED) in which students enrolled in my section of LL ED 005 College Reading were also enrolled in a section of PSYCH 100 Introductory Psychology. The purpose of the pairing was to provide concrete opportunities for learning the expectations, strategies, and skills of college level reading through connections to a real-time content course. My hope was that students would more easily see the relevance of the work we did together in the reading course and that the pairing would provide distinct moments to discuss and change behaviors and attitudes that would lead to a higher rate of academic success. GPAs for students in all sections will be tracked over two semesters. An IRB proposal was submitted and approved and student informed consent was gathered before the start of the project. A presentation proposal has been accepted for the spring 2013 Lilly Conference in Bethesda, MD, Carpe Diem: Making the Most of a Paired Reading Course for First Semester Students.

The course was divided into three main chunks: information processing, meta-cognition, and critical thinking. My thought was to give students the jolt they needed right away to get up to speed with the amount of reading needed and also the amount of independent learning required. So students did a lot of information processing activities like Cornell notes, index cards, chapter review sheets, online quizzing, and concept mapping. After they successfully completed the first exam, we moved onto deeper processing skills and critical thinking. My assumption was that they would continue with the strategies and activities that we did in the initial units throughout the course.

In addition to the strategies and tools we used to help students learn the psychology content, we did a lot of work through toolbox assignments (combination of activities and self-reflection on a given topic) on behavior and attitudes that help or impede academic success. Curiosity level inventories, confidence levels, decision-making styles, point-of-view reflections, significant learning traits, and study environment scans were some of the toolbox assignments that students completed. My goal for this portion of the course was to raise student awareness about their own traits and abilities making it possible for them to better control many factors that correspond to success in college.

I learned a lot through this process… from IRB and the perils of a proper informed consent to include FERPA…to getting enough data/information to understand (as much as you ever can) what’s happening in a classroom, to using that analysis to make changes that make a difference. Overall, I think the course was a success with some important caveats going forward. These are the changes I’m proposing:

  • Students paired with only one psychology instructor – even though the instructors used the same textbook, they were on different chapters – so it made it impossible to have a real common experience of learning psychology – we were never (literally) on the same page
  • Students should enroll only in the daytime class of psychology – the once a week, 3 hour evening class was too big of a leap for first semester students – requiring too much independent learning and discipline
  • I had them doing intense info processing (Cornell notes, completing chapter objective sheets, study plans, practice quizzes, review sessions) for only the first 2 exams – I’ll readjust to do this longer – through exam 4 – and then release them to their own power
  • Not use the TP text – it was enough with just the psych text.
  • Data to collect (and include on the informed consent form for them to sign): gpa for 2 semesters, course grades, Psych exam grades, SAT reading scores, end-of-semester survey, clicker question responses

The greatest benefit of this project in my mind was how quickly we could start to talk about expectations of college work and start to address that part of the high school to college gap more quickly through a common experience (learning the psychology content) – Overwhelmingly, students reported a very positive experience in the course. In the next iteration, I want to make sure that it is also more effective in terms of psychology outcomes, through the changes I am proposing.

Future Iterations

Combining Retention, Student Support, Learning Analytics in a College Reading Course
Recording – https://meeting.psu.edu/p5xe3ht92go/

In a future iteration of the course, I hope to combine the Effective Lifelong Learning Inventory (ELLI) + Reflection + Learning Analytics within the college reading course. Thanks to Simon Buckingham Shum,(Knowledge Media Institute of The Open University, UK) who mentioned his work with the ELLI project, I believe I have found a very interesting project to try with a future iteration of LL ED 005 college reading students!

Last fall, I worked with students in reflective activities called “toolbox assignments” that asked them to take different inventories (curiosity for example or successful choices) and then reflect on their scores and what it means in terms of their learning/academic success. It was pretty piecemeal…and nothing electronic, so I couldn’t really (without a lot of work) “see” patterns/trends… so when I saw the ELLI inventory, with its 7 dimensions of lifelong learning (see below) – along with an example which showed how mentors/mentees were using it in a reflective process to discuss student scores on each dimension and look at growth/change over time…and now with the learning analytics piece – thinking about having students take the inventory – reflect in a blog – and track with learning analytics to analyze – using the WordPress Blogger Enquiry plug-ins, we get more data! This could be a really interesting project that makes a lot of sense in this first year course for students!!!

Here is more information about ELLI and a short bibliography…ELLI (Effective Lifelong Learning Inventory – a self-report 72 item web-based questionnaire – internal structure was factor analyzed and validated) measures how people feel about themselves in a particular domain (7 total) at a given point in time

  • Changing & learning
  • Critical curiosity
  • Meaning-making
  • Dependence and fragility
  • Creativity
  • Learning relationships
  • Strategic awareness

Links
Crick, R.D., Broadfoot, P., Claxton, G. (2004) Developing an effective lifelong learning inventory: the ELLI project; Assessment in Education; Vol. 11, No. 3.

Crick, R. D. & Yu, G. (2008): Assessing learning dispositions: is
the Effective lifelong learning inventory valid and reliable as a measurement tool?  Educational Research, 50:4, 387-402.

Ullmann, T.D., Ferguson, R., Shum, S. B., Crick, R. D. () Designing an online mentoring system for self-awareness and reflection on lifelong learning skills. pp. 34-42. In: Proceedings of the The PLE Conference 2011, 10th – 12th July 2011, Southampton, UK.

Enquiry Blog Builder –

SP 2011 – Embedding a Tutor

This semester, my PSU ESL class was canceled due to lack of enrollment.  Because of that, I have time to focus on a new project: How does embedding a tutor, and differentiating instruction at the lowest level of the course sequence, impact student success?

Teaching the 026 level (entry level) of students has been challenging. Because it is the entry level, we get a great range of student skills. It is at this level that those students with lower literacy or academic preparation, or students from language backgrounds vastly different from English can really struggle. We have been really worried about those students who just can’t seem to stay with the rest of the group… and with the pressure of readiness for 027 and beyond, teachers can’t stop to make sure that all students keep up.

The college has tutors (especially for ESL) and other resources, but not all students can get to campus out of class time. So we discussed the possibility of embedding a tutor into this lower level class to see if differentiating instruction could really make a difference.

Here is what is happening so far. The tutor is a second-year bi-lingual student, very accomplished academically and interested in helping. I’ve started planning lessons with sections that have us all working together and then breaking out into sub-groups to either extend or review concepts. We have several students who are repeating the level so we can compare their progress to that of last semester. I also plan to use similar tests so that we can compare scores last semester and now with the embedded tutor in place. Below is the first differentiated plan and accompanying handouts

ALLBook & CD CheckLesson:
How to Use your Textbook/ CDs – http://eslyork.wordpress.com/026-2/lesson-2-how-to-use-your-textbooks-present-tense/How to Learn Vocabulary + word listsEvery Word has a Job to do!http://eslyork.wordpress.com/026-2/lesson-2-part-two-present-tense/
PracticeIdentify subjects & verbsWord Order – re-order Practice with TutorIdentify nouns (pronouns) and verbs handoutSentence strip activity (word order review)
ALLLesson – Present Tense – WHEN to use & HOW to use it
PracticeCorrectly forming the present tense with nouns and pronounsWorksheet on Present Tense Practice with TutorCorrectly forming the present tense with nouns and pronounsWorksheet on Present TenseGet them to NOTICE  facts/information and routines… that’s why we are using present tense
ALLPronunciationVowel Supplements  12 – 14 – pages 187 – 194 as time permitsHomework overview

Handouts

EmbedConjugationPractice

TUtorEmbedGroupActivityIDNounsProVerbsWordOrder

EmbedGroupActivityIDNounsProVerbsWordOrder

IDsubjectVerbWordOrder

Debrief of first differentiated session

Our tutor is a dream! He is helping in 2 ways – 1) during the break-out sessions with differentiated instruction – he worked with a group of 3 (mixed ability levels) on a sentence strip activity and categorizations to identify nouns, pronouns, verbs and eventually subjects of sentences. I worked with the other group on the same goals, just a more challenging delivery and activities. We switched groups half way through the activity. Our tutor  was  invaluable – He jumped right in and it worked out great – 2) during in-class activities, we can both get around the room and check student work.

Immediately I see one benefit – More one-on-one feedback more often during the lesson. We were able to catch a lot of misunderstandings (especially about process). This class size is perfect. There were 8 students there and they all got a lot of individual attention as needed. Another benefit – they are getting to know the tutor which might perhaps make them more apt to work with him at other times. When he told them about his tutoring hours before class, they were very interested!

We used the extra 2 tables in the back of the room during the breakout session.

I think the mixed ability groups might be the way to go in the beginning so that people don’t feel labeled as “slower”. I don’t want them to lose confidence about themselves! Also, the fact that he and I switched groups also helped them to think that the groups were separate but equal (maybe). We are monitoring this as we go along – VERY interesting process and I am grateful to our tutor – a super partner in the classroom!

Let’s see how they do on the quiz next week. Present Tense (we didn’t tackle “be” yet) and identifying subjects/verbs.  Last semester it was clear that many didn’t understand conjugation, even after many repetitions – that was the thing I wanted to tackle in this lesson – to set the stage for later work with tenses.

FA 2010

I made extensive changes to all of my courses (what was I thinking?) because I just felt the time was right to really bring it all together – my coursework in linguistics, independent study on teaching grammar, and recent experiences with multiple ESL levels in the classroom gave me lots of food for thought that I was eager to implement.

ESL 4 Academic Writing – this course got a significant face lift which turned out to be something I am really happy with. It took a few semesters to really fine tune the course to match student needs and really target the needed skills – both for language and academic skills… Here are the changes in a nutshell

1) Vocab development – continued to give pre- and post-test to mark progress. Held them more accountable with weekly vocab notebook entries and weekly quizzes based on their required workbook chapters. Increases from pre- to post-test of 7.5%, 19%, 44%, 52%, and 57%!! Students also noted in SRTEs that they liked the weekly quizzes and felt they helped them!

2) grammar dev – this has been a roller coaster ride over the years… I finally feel like I got a good balance and with the help of some great books on teaching grammar specifically to ESL students, and thinking about grammar needs specifically in academic writing, I think this part of the class has vastly improved… although no where near the gains for vocab…

The first part of the course was a pretty intense grammar review with lots of practice and homework using a great book that I now have as a required text for the class – nice to find a good match finally!!! – Think About Editing: A Grammar Editing Guide for ESL Writers by Allen Ascher (Heinle & Heinle 1993)

3) Writing – at the same time that we are doing grammar review and they are working on their vocabulary (independently), we begin writing almost immediately and try to incorporate what we are doing into the writing. There are several writing tasks that we do durin ghte semester – First – how to write an e-mail to a professor…. Next – we work on several types of academic structures (paragraphs only) – we were able to do chronology (Pompeii), process (digestion), and a basic summary type writing using questions to help formulate the answers, focusing on writing essay test questions from a piece of content (types of clouds). Finally, they kept a learning journal in a blog that they wrote in after each major writing assignment – describe what they learned about their own writing and writing in general that they could apply to other courses.

*******************************************************************

Spring 2010 – Course in College Teaching Action Research Project

This semester, I am facilitating a local offering of the Course in College Teaching. One of the projects we are completing is an action research project with three milestones – project rationale, implementation, and analysis & evaluation. I think it is a good exemplar of a process that can be very useful when trying something new in the classroom. How can we evaluate the effectiveness of it?

Milestone #1

Background: I’m teaching speaking and listening exclusively in a new course – Usually, ESL is taught with all 4 language areas integrated – listening, speaking, reading, and writing… but they are divided in this current program… so I have to really think through how to do this now. Another teacher is doing the reading/writing portions and we have tried to coordinate to support each other… but it still comes down to changing fundamentally my approach which is usually highly integrated….This is a high beginning ESL academic prep course
So week one, I drove everyone crazy doing a LOT of pronunciation work – la la le le lo lo – OK ENOUGH! I’m driving myself crazy… Time to get back to the books… What does research say about teaching pronunciation and improving listening skills…My goal for them is to really ACQUIRE the language components we are working on – that means they internalize them and use them without having to give much thought to them… so what does research say can help to make this happen?According to David Nunan in his text Practical English Language Teaching (2003), language is acquired when students have the opportunity to learn it in context… that is – isolated anything – grammar, vocab, pronunciation – might be remembered temporarily – might even be answered correctly on an exam – but it won’t become THEIRS until they understand the contexts in which it is used and get enough practice with REAL USE of the language in context that it starts to sound and be normal for them to use….So my question is this: How can I design the activities I do with my students this semester to be contextual in nature and not simply isolated activities… and will they really acquire the elements we are working on in the end….
***
***

***

Milestone #2
So what data can I collect to help me answer this question?

I started by doing a lot more background reading on my chosen model – SIOP – Sheltered Instruction Observation Protocol http://www.cal.org/resources/digest/sheltered.htmland realized it was a really good fit for what I was trying to accomplish and has a good track record from a research standpoint. So my strategy is to apply the model to an upcoming lesson on present continuous tense (using the workbook in a unit on hurricanes)From the Center for Applied Literacy, ” The Sheltered Instruction Observation Protocol (SIOP) Model* is a research-based and validated instructional model that has proven effective in addressing the academic needs of English learners throughout the United States.”(http://www.cal.org/siop/about/index.html)I chose this model because it is academic English (rather than survival English ) focused and because I felt it would help me to accomplish the following:

  1. to have students be more actively involved more of the time (less teacher talk – more student time on task)
  2. to have the activities we do be more contextual in nature – not isolated activities – but embedded into a meaningful context – remember Nunan (2003) – language is acquired when students have the opportunity to learn it in context…
  3. to have the assessments I use be more performance based – so I can see to what degree they are acquiring the language chunk we are working towards…
  4. to get away from just using the workbooks without any careful planning about how to use them effectively
The overarching goal is to create lessons whereby students actually acquire the target language chunk!I also realized just how much my overall goal, strategies/assessments/ question / data collection etc… were refined with time and working through the process!!!So my strategy is to use the SIOP Lesson Planning model as my guide in designing a lesson and then gather data to see how well it helps me to achieve the goals and targets I stated above. The lesson is on the present continuous tense and takes place within a content unit on hurricanes.I used the strategies and suggestions for each step in the model – See the lesson plan I designed to use with my class in its entirety here -SIOP Lesson PlanPresentContinuousTense.docAfter working through each step of the model and incorporating strategies for each part – I really could see such a difference in the overall DESIGN of the lesson – and I could see so many elements in the workbooks themselves that include the important aspects of this model – but I was blind to them, so I couldn’t capitalize on them.Data Collection
* Teacher observations – as to time on task – how much teacher talk versus student interaction – compare these to a previous workbook driven lesson
* Use of assessments  (informal) to determine level of understanding – handouts, authentic activity, thumbs/up/down/fist, question cube
* student survey – Do they feel they can successfully use the PCT and also tell when others are using it? Do they feel they need more practice or are ready to move on to the next topic?
* Next session – check for retention after review – follow-up quiz – Did they retain it – in conversation – can they use it in real dialog? Can we move onto the next language chunk?

Reflection

I am trying it out tonight with my class – I feel really well prepared, not only by WHAT I am going to do… BUT WHY!!!  It is a really good thing after feeling like I was just in survival mode for a few weeks – Now I feel much more ready to keep working with them effectively so they are better prepared for the next level.The first time working through the model took a lot of time – and several iterations before I cross-checked for my goals and all the elements of the model. I am sure that each time I work through this in the future, it will go much faster and eventually I won’t need to look at the model in such detail – Although I was grateful that the info I found not only had the important elements, but also activities and strategies to use with each part of the model… so that was a HUGE help!!Overall, I am really happy to have the opportunity to hone my knowledge and skills about what it means to teach academic language to non-native speakers. I am now going to try this model with my higher level class – although that one has been going much better – their level is much closer to what I’ve been used to lately….I also feel much more ready for when I am to be observed sometime in March – Much more able to answer questions about what I’m doing and why… and to have a more meaningful conversation with the observer – to improve what I’m doing even more…Frankly, what I feel most is relieved…..and a bit more confident… and I hope to feel satisfied by the end of class :)
***
***
***
Milestone #3 – Implementation Analysis, Evaluation, & Next Steps
What an eye opener this was for me!!!  While I’ve done this type of process (implementing a change or classroom improvement) in a very loose fashion over the years, I’ve never actually gone to the trouble of thinking about (and collecting) data and it has really given me a lot to think about.After implementing the change, overall, I feel more confident that the time spent in instruction was more efficient and effective and I believe I will continue using the tool I implemented throughout the semester – I can see this being especially useful when teaching a new content area or something in a new way – because it gave me a way to stay focused on the learning goals and a way to design that drives everything towards acquisition…I could also see how easy it is to get lazy in a sense after having done something so many years – the danger with this is that you forget that your brain (which has now mastered the material – whatever it is) is taking all these shortcuts – or rabbit trails down more interesting paths – for me) and forgetting that I need to be focused on getting my novices tot he same place – and that takes careful planning and putting myself in their shoes once more – easier to just go in there and wing it – but effective? I don’t think so.This quote from the handout with the SIOP Model sums up a lot for my experience, “When students spend their time actively engaged in activities that relate strongly to the materials they will be tested on, they learn more of the material.” http://www.misd.net/bilingual/ELL.pdfMy observations
Several things stand out immediately that were a direct result of using the SIOP model

  1. Less teacher talk, more time on task – Students were actively doing something related to the target language element 100% of the time – so I carefully planned so that even while I was showing them something or talking, they were doing something with the material and not passively sitting there listening. I heard myself saying, “Take your notebooks out – everyone should be writing this down.” and then we moved right into activities in different groupings – pairs, and then sections of the class, and then whole group.
  2. More opportunities for students to practice with feedback – When I look at a previous lesson – using only the workbook – I was surprised to see that while there were many activities that we did, only 1 was directly related to the language target – It seemed like so much more!  In the planned lesson last night – there were 6 different activities focused specifically on that target – 2 writing, 3 speaking, and 1 listening – 5 of which I needed to create to give them practice in many different ways.
  3. Identifying, reflecting on, and then designing for certain outcomes – makes them happen!
    I wanted to make sure that the goal was that they would really be able to use the target language element in an authentic setting – so I planned for it, used the model to design enough practices, and then designed a final activity asking them to do it.. and they did!!!
  4. Much more focused and efficient lesson!!!! – When I looked at the plan, I thought, “We’ll never get through all this in the time we have!” Based on how other classes have gone… but I was SHOCKED when we had finished, I gave them the survey, everyone seemed pretty confident with the skill… and it had only taken an hour!!! We still had an hour plus break left! – So, I realized how inefficient my other lessons were – and careful planning had made them not only get through more material – but more effectively too!!!

Other data
Survey – at the end of class, I handed out a very simple (language level appropriate) survey asking 5 questions, with yes, no, maybe responses…

  1. Did the handouts help you to learn the present continuous tense?
    50% YES, 50% MAYBE
    I think the maybe votes were because they were hesitant to come across so positive and confident about what they could do. (or the handouts need work – but I don’t think so in this case) and the assessments showed they could do it.
  2. Do you feel you can correctly use the present continuous tense when writing?
    67% YES  33% MAYBE
  3. Do you feel you can correctly use the present continuous tense when speaking?
    67% YES  33% MAYBE
  4. When listening, can you tell when people are using the pct?
    67% YES  33% MAYBE
  5. Do you feel you need more practice with the language target…
    50% YES,  33% MAYBE,  17% NO
    This tells me we need to continue building their confidence with this skill – so we’ll review it next week and practice several more times so it really starts to stick for them and they know it.

Assessment info

Final activity – where they had to use the pct in  practicing a real phone call to a neighbor showed that they could all do it, given an authentic scenario.

Interesting though – in one of the other handouts – some people could answer the “academic” questions – a picture of a woman cooking and asked – What is she doing? Response rate – 100% “She is cooking”… but when the question asked simply, “What are you doing?” with no picture, and I pointed to them and said – What are YOU doing?” several wrote, “I am write”.. and I had to get them to notice and then correct the error – which they did and then were able to do it correctly in the phone call example at the end… so interesting to me!!!  If they only had had the one activity for practice, as in the earlier lessons, it would not have been enough to really acquire the element…

TO be continued…..

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>