An Empirical Methodology for Investigating Geocognition in the Field

Download PDF


Heather L. Petcovic¹, Julie C. Libarkin², Kathleen M. Baker³

¹Department of Geosciences and the Mallinson Institute for Science Education, Western Michigan University, 1187 Rood Hall, Kalamazoo, MI 49008; heather.petcovic@wmich.edu 
²Department of Geological Sciences and Center for Research on College Science Teaching and Learning, Michigan State University, 206 Natural Sciences Bldg, East Lansing, MI 48864; libarkin@msu.edu 
³Department of Geography, Western Michigan University, 3219 Wood Hall, Kalamazoo, MI 49008; kathleen.baker@wmich.edu

ABSTRACT

The investigation of how geologists engage in field mapping, including strategies and behaviors, is an open area of research with significant potential for identifying connections to best instructional practices. While study of experts in an array of disciplines has yielded general conclusions about the nature of expertise, the consideration of geoscience experts, especially in authentic settings, is virtually unstudied. Field mapping involves a complex interplay between the individual mapper and the natural environment. Both cognition and behavior influence the observations and interpretations that ultimately yield the map, a representation of the natural world. We set out to establish a methodology, adapted from existing studies of expertise, that would allow us to document cognitive and behavioral processes involved in situated mapmaking and generate preliminary insights into expertnovice differences in mapping behavior and cognition. We present here a theoretically–driven, mixed methods methodology, and suggest that navigation coupled with field artifact and audio data provide the richest and most meaningful insights into geocognition in the field.

INTRODUCTION

The best geologist is the one who has seen the most rocks.

(Anonymous) 

What does it mean to be the best, most expert geologist? Is it simply, as the saying goes, the ability to recognize the most rocks? Or, does expertise differ in more subtle or complex ways? The nature of geological thinking (“geocognition;” Libarkin, 2006) is both poorly understood and poorly studied from an empirical perspective, although numerous investigators have discussed what geoscience expertise might look like (e.g., Frodeman, 1995; Raab and Frodeman, 2002; Ernst, 2006; Kastens and Ishikawa, 2006; Petcovic and Libarkin, 2007). 

Only recently have researchers begun to systematically investigate how geoscientists engage in authentic practice, in both simulated and real settings. Bond and colleagues (2007) show that prior knowledge and experience play a large role in conceptual uncertainty, essentially biasing expert interpretations of seismic sections. Kastens et al. (2009) discuss the relationships between novice and expert mapping behavior and interpretation of artificial rock outcrops, finding clear differences between how these two populations see and understand these phenomena. In one of the few studies of expert behavior in a natural field setting, Gahegan and Brodaric (2001) provide evidence for the influence of situated cognition in map generation. Other forms of geologic observation and data interpretation are probably also laden with interpretive, situational, and experiencerelated distortions. Given the paucity of existing studies in natural or contrived settings, we are forced to make assumptions about the nature of geological expertise, and we still know very little about how novices (undergraduate students) become experts (professionals), what skills are important to gaining expertise, and how traditional education shapes geocognitive development.

Geoscientists universally regard field mapping as necessary for the development of expertise in the geosciences (MacDonald et al., 2005), and geologic field mapping courses play a significant role in college and university departmental curriculum and expectations (Manduca and McDaris, 2007). Anecdotal evidence suggests that this a priori assumption exists across geoscience disciplines, almost regardless of the importance of field work within that discipline. Within any geological sciences discipline, nearly all postsecondary students receive some fieldoriented training, typically as a semester course in field methods, and/or a four to six week summer field course. Despite this infusion into the curriculum, the nature of expert and novice fieldbased geocognition, as well as the interaction between cognition, behavior, and environment during field activities, is open for debate.

By utilizing data and method triangulation, we propose to establish a methodology for investigation of field mapping behavior and cognition. In social science research, triangulation refers to a specific mixed methods research design in which qualitative and quantitative data are gathered and analyzed concurrently yet independently, so that the results of each data set can be compared to one another (e.g., Creswell, 2009). The triangulation of multiple data sets collected by one or more observer, use of competing theoretical schema, or application of multiple methods to analyze collected data are all insightful mechanisms for enhancing the rigor of research findings. This is particularly true in the cognitive or behavioral sciences, where interpretations can be biased towards preconceptions, and where all aspects of validity and reliability can be difficult to assess (e.g., Golafshani, 2003).

Our approach utilizes multiple qualitative and quantitative data sets, with multiple methods applied to investigate our quantitative data. Specifically, we collected and analyzed selfreports of prior geology experience, field maps, and navigation data from all participants, and ThinkAloud audio logs gathered during mapping plus a followup interview from a subset of participants. We suggest that this mixed methods approach offers richer and more robust understanding of field cognition and behavior than that available through simpler single data set analysis. We developed and tested this methodology at two field sites in the Michigan Upper Peninsula, USA, using data collected from seven participants; here, we describe the methodology and results of this study, and make recommendations for using these methods to study authentic geocognition in the field.

BACKGROUND

Expertise as a generic construct has been investigated in a wide variety of disciplinary areas such as chess, history, teaching, and physics, providing some general insights into the nature and development of expertise, as well as into the application of expertise studies to education. Experts display rapid recognition of patterns that are common in their disciplinary area, thinking that revolves around big ideas (rather than memorization and recall), interrelated knowledge, the ability to self-monitor and evaluate progress on a disciplinary task (metacognition), flexible thought processes, and awareness of complexities and abstractions (Table 1 and references therein). This body of research suggests that explicitly attending to aspects of expertise, including how experts solve problems, can align student thinking closer to expert practice. For example, research in physics education shows that experts differ from novices in having knowledge that is hierarchically organized around key principles, and a highly metacognitive approach to problem-solving using such principles (e.g., Chi et al., 1981; Van Heuvelen, 1991). Yet while useful for understanding the big picture, these expertise studies are often focused on nonscientific disciplinary areas, and those few studies that look at scientists rarely venture into authentic lab or field settings.

Within the geosciences, novice studies far outstrip investigations of experts, and in the few studies of expertise theoretical arguments outnumber empirical research. We do know from philosophical work that the geosciences are much more probabilistic, historical, reliant upon perceptual judgments, and influenced by intuition and imagination than other physical sciences (Raab and Frodeman, 2002). Certainly, geological interpretation is by nature historical and uncertain, as geological reasoning reconstructs the most probable sequence of events producing landscape features or rocks. Geologists in the field are physically embedded within their research context, thus geological interpretation can be highly subjective (Raab and Frodeman, 2002). This is different from more “objective” laboratory sciences (such as chemistry) where the body of the observer is removed from the experiment, but is probably a shared characteristic with other fieldbased disciplines (such as ecology or anthropology). 

TABLE 1. ATTRIBUTES AND BENEFITS OF EXPERT CONGNITION

CHARACTERISTIC (Experts have…)1	BENEFIT TO EXPERTS1	POSSIBLE IMPLICATIONS FOR GEOLOGICAL FIELD WORK
Recognition of patterns in information due to familiarity with subject	Recognition of patterns triggers additional knowledge that is relevant to specific task	Experts rapidly recognize rock types based on just a few key features. Experts may also have more facility in picking out largescale structures on geological maps.
Thinking revolves around big ideas, rather than memorization and recall	Rapid application of concepts important for problemsolving	Concepts related to plate tectonic, such as characteristics of divergent or convergent margins, may be called upon to explain field observations.
Knowledge that is interrelated (chunking)	Efficient storage and recall of large pieces of information. Efficient problemsolving.	When describing a rock sample, an expert might call up an interrelated body of knowledge concerning rock texture and classification.
Ability to selfmonitor and make decisions based on understanding (metacognition)	Critical to development of effective problemsolving strategies, including backtracking	A field geologist may use metacognition to track his progress in a new area; new data may prompt return to a previous location to doublecheck a rock type or relocate a contact.
Flexible thought process	Enables integration of knowledge from multiple domains	Field work in geological mapping may require the ability to integrate knowledge from petrology, structural geology, stratigraphy and sedimentology, paleontology, etc.
Awareness of complexities and abstractions	Results in structural, rather than surficial, knowledge and recognition of complex connections between concepts	Field mapping require ability to interpolate data between rock outcrops and to mentally develop a complex, 3D representation or rock distribution.

1Modified from Petcovic and Libarkin (2007) and synthesized from Bransford et al., 2000; Donovan and Bransford, 2005; Feltovich et al., 1997; HmeloSilver and Nagarajan, 2002; Chi et al., 1981; Chi, 2006; and Hoz et al., 2001.

Empirical studies of geoscience experts suggest that 2D and 3D visualization abilities are critical to solving geologic problems (e.g., Anderson and Leinhardt, 2002; Abel et al., 2004), and visual extrapolation of features in rocks and landscapes plays a major role in geological interpretation (Abel et al., 2004). This ability to extrapolate visually may be tied to spatial ability as suggested by work in geography. For example, expert topographic map readers move fluidly between maps, the landscapes they represent, and the 3D subsurface structures underlying the landscape, whether actual landscapes or images (e.g., Pick et al., 1995; Eley, 1991). Geographers have long investigated the nature of mapreading and mapgeneration expertise using a wide range of data and methods (e.g., Lobben, 2004; Barkowsky and Freksa1, 1997), and recent work has begun considering mapping behavior in simulated geological settings (Kastens et al., 2009). Similar investigation of map development in authentic field settings is rare, although at least one study (Brodaric and Gahegan, 2001) provides empirical evidence for changes in field cognition in response to environmental inputs. In other words, geocognition is situated (e.g., Wilson, 2002). Brodaric and Gahegan (2001) also identified the importance of prior experience and knowledge on the development of geological field categories, as suggested by nonempirical discussions. 

Methodologies commonly employed by science educators to investigate expert cognition and knowledge structures can be subdivided into four categories of empirical cognitive experiments (recall, perceiving, categorization, and verbal reporting; Chi, 2006). Although these techniques are specifically used in laboratory settings, they can be applied with some modification to authentic settings. For the investigations of field behavior, verbal reporting and perceiving tasks are the most useful and easily generated data sets; in addition, behavior as a reflection of underlying cognition can also be used to generate an understanding of cognition. For example, spatial data of human movement (i.e., navigation) can be used to infer underlying intent and cognitive process (e.g., Schlider, 2005). In general, navigation data will be most useful if used in combination with data that can be directly correlated to reasoning. 

Verbal reporting and perceiving tasks provide direct observation of subject thinking. Verbal reporting as a specific category of cognitive experiment refers to participant explanations as they engage in a realtime task. These may be ThinkAlouds, such as those performed during development of many concept inventories (Zeilik, 1999; Libarkin and Anderson, 2007), as explanations of tasks, or during conceptual interviews (Hoffman et al., 1995). ThinkAlouds can be used to ascertain both the validity of the underlying task as a cognitive prompt, as well as differences between expert and novice cognition. Perceiving tasks focus on the filtering that occurs as an environment or artifact is perceived, considered, and related to ongoing cognition. These types of data, whether ultimately resulting in verbal reports or artifacts such as notes or drawings, can reveal differences between expert and novice cognitive processes. For example, expert and novice teachers viewing and talking about a video of classroom instruction showed differences in the complexity of observed patterns as revealed in ThinkAlouds (Sabers et al., 1991). Similarly, research on medical students and professional doctors reveals differences in the amount of material that experts and novices take note of, as well as in the specific material that is considered of interest (Patel et al., 2002). 

Navigation behavior can be quantitatively recorded through use of a Global Positioning System (GPS) unit attached to a subject moving through a natural space. Analysis of GPS tracks provides a mechanism for assessment of natural movement (Turner and Penn, 2002); for example, pedestrian wayfinding behavior as revealed by GPS suggests interesting relationships between individual and group movement decisionmaking (Raubel, 2001). Models of natural human movement and geospatial reasoning are computationally intensive when left openended (Parunak et al., 2006; Turner and Penn, 2002), so construction of an activity ontology (Kuhn, 2001) is often used to identify participant actions throughout the track. For example, an activity ontology could consist of orientation (broad surveying of landscape), movement (rapid or slow progress), and inspection (remaining in one spot). GPS units have been applied to tracking of college undergraduate student movement during geologic mapping by Riggs et al. (2009), suggesting influences of environment on mapmaking behavior.

METHODS

The methods used to engage participants and collect data on field cognition and navigation are described below. We have provided sufficient detail to ensure effective reproduction or modification by other groups. 

Research Design – This pilot study was designed to develop and test methods for eliciting cognitive and navigational processes used during geologic field mapping. The original study design included three volunteer participants with varying prior experience in geologic mapping recruited from among colleagues of the researchers; participants mapped at two field locations in Michigan, USA. Data collected from these participants included a background and experience questionnaire, field maps and notes, a GPS track, a ThinkAloud audio log, a photographic log, and a followup interview. 

A purposeful sample of four students from a college undergraduate field methods course that used the same field locations was later added to the study, in order to obtain more GPS tracks and maps for analysis. However, because of time constraints in the field course, we were not able to collect audio log and interview data from the student subjects. While the nonequal treatment between these groups (volunteers vs. students) certainly affects the outcomes of the project, our original intent was to produce a viable research method and preliminary hypotheses on noviceexpert field cognition that could be tested in the future with a larger population of subjects and more controlled research design. 

Location - This study utilized two field locations near the town of Marquette in the Michigan Upper Peninsula, USA. Rocks exposed at both sites are distinctive enough that a detailed knowledge of the local geology and stratigraphy was not crucial to effective mapping. At Lighthouse Point (LP) all participants mapped crosscutting relationships between three intermittently exposed rock units onto a simplified topographic base map. The mapping area was 145 meters x 190 meters. At Harvey Quarry (HQ), all participants mapped a crosssection through a structurally complex syncline containing seven rock units. Rocks were intermittently exposed over a horizontal distance of about 450 meters, with the site flagged every 30 meters. Exposure consisted of vertical quarry walls and road cuts; participants drew rock units on a simplified outline of the traverse and were instructed to project rock units and contacts below the ground surface to produce the crosssection. All participants (volunteers and student sample) used the same base map and crosssection outline.

Participants - Seven participants representing a continuum of prior geologic study and field mapping experience ranging from none to 10+ years took part in the study (Table 2). Participants were initially ranked along the expertnovice continuum based on selfreported prior geologic mapping experience obtained via a written survey (Geologic Experience Survey; GES). Questions on the GES elicited information about participant demographics (age, gender, race, etc.), education, work experiences, and prior field experiences. Criteria for placement along the continuum included: (1) Number of undergraduate and graduate geology courses taken; (2) Number and level (BA/BS, MA/MS, PhD) of degrees held in geology; (3) Number, length, and purpose of prior geologic field experiences including field courses, fieldbased thesis or dissertation research, fieldbased work or internship experiences, and teaching experiences; and (4) Years of professional mapping experience, where applicable. Two of the authors reviewed these data and agreed upon the relative level of geologic expertise associated with the backgrounds of each participant. Based on these criteria, we classified participants as five novices (N1 through N5) and two experts (E6 and E7), with participants ranked from most novice to most expert (i.e., N1 had the least geology background and prior mapping experience; E7 the most; Table 2).

Data Collection – The three volunteer participants, including one novice (N2), and two experts (E6 and E7) (Table 2), mapped first at Lighthouse Point and then at Harvey Quarry on two consecutive days. These participants were directed to simply create a bedrock geologic map (LP) or crosssection of the traverse (HQ), to record their thinking during the mapping task with the audio recorder, and to take pictures of interesting features during mapping. Participants were allowed unlimited time, although all volunteer participants completed each task in under 4 hours. Two continuous GPS track records (modified from Riggs et al., 2009) were used to validate the most appropriate unit setting, with one GPS set to collect one waypoint per minute and the second set to collect data automatically, more frequently when the wearer was moving rapidly and less frequently when the wearer was stationary or moving slowly. Semistructured followup interviews were intended to clarify maps and specific map features, and capture participant reflections of the mapping experience. A set of openended probes was used to initiate the interview process. In addition to questions designed to clarify specific map features (e.g., “what does this symbol on your map mean?”), probes included the following questions: 

TABLE 2. PARTICIPANT DEMOGRAPHICS AND PLACEMENT ALONG THE EXPERT-NOVICE CONTINUUM

Is there anything you would like to tell us about today’s activity or your map?

Which parts of your map do you feel most confident about? Least confident about?

Can you give us an overview of the strategy you used when you did the mapping?

Imagine you have another day at the field site. Where would you spend your time, and why?

Imagine you were doing actual research in this area. How might that work differ from what you did today?

Please take a moment and look at your photos. Can you show us approximately where the most important photos were taken and discuss why you took them? 

The remaining subjects (N1, N3, N4, and N5) mapped both field sites one day prior to the volunteers, as part of a twoweek, undergraduate field methods course. None of the researchers or volunteer participants were instructors of record for the field course, although one of us (HP) had accompanied and taught portions of the course in past years. Mapping activities took place about midway through the course, after an introduction to the regional stratigraphy. At both sites, students were explicitly instructed to interpolate lithologic contacts through cover and cultural features, to collect strike and dip of bedding in 510 locations spread over the map area, to identify map units by stratigraphic name, and to write a complete description of all map units. The course instructor pointed out several key features and locations to help students begin mapping at the start of the day. Students were allowed an entire day (~8 hours) to complete their map plus additional requirements (e.g., lithologic descriptions); in general, student participants completed each project in 46 hours. The two sites used in this study were the students’ first graded mapping projects, and were completed independently. 

Data Analysis - Participant maps (LP site), crosssections (HQ site), and GPS tracks were analyzed quantitatively using ArcGIS 9.2. Audio logs and interview data were analyzed qualitatively using an emergent coding scheme in which themes were not predetermined but instead emerged from analysis of the data (e.g., Patton, 1990). Participant maps were scanned, digitized, and converted to GIS files (Figure 1). Based on our initial examination of the maps, we found several parameters to be the most useful for making comparisons between individual participant maps and for identifying common aspects among maps for experts vs. novices: percentage of area mapped as specific lithological units (rock types), number of mapped units, and average number of mapped polygons per unit. 

All GPS tracks were downloaded into GIS and each individual track was examined for total time spent mapping, total distance covered, and average speed. For the Lighthouse Point site, each participant’s track was superimposed onto his or her map, and a kernel density map, which assesses spatial patterns of time spent mapping various areas, was created (Figure 1). Relative time spent mapping particularly sensitive areas, such as areas with evidence of crosscutting relationships or contacts, was investigated via this kernel density analysis. A generalized path for each track was created to examine the number of overall loops and retraced paths in an individual’s overall mapping strategy, thus establishing individual track complexity (Figure 1). Finally, all tracks for the Lighthouse Point site were compiled into a single kernel density map to establish evidence of trends across the entire participant group (see online supplement). We did not perform this analysis for the Harvey Quarry site as the location required a horizontal traverse and mapping of vertical outcrop to produce a cross-section.

FIGURE 1. Sample Lighthouse Point data set from subject N5, a student. The entire map area is about 190 m eastwest by 145 m northsouth, and north is toward the top of the page. (A) Generalized GPS track; arrows show direction of travel. (B) Kernel density map created from the GPS track, showing relative time spent in specific areas. (C) Scanned and digitized geologic map.

TABLE 3. PERCENTAGE OF TOTAL MAP AREA ASSIGNED TO EACH UNIT AT LIGHTHOUSE POINT

PARTICIPANT	SCHIST, GREESTONE, AMPHIBOLITE	BASALT, GABBRO (%)	METARHY0LITE, FELSIC PORPHYRY (%)	OTHER ROCK TYPES (%)	UNMAPPED AREA (%)
N1	54	13	1	0	32
N2	0	71	14	16	0
N3	27	39	2	0	31
N4	36	33	3	0	27
N5	29	39	1	0	31
E6	73	15	1	0	11
E7	76	19	5	0	0

Participant audio logs and interviews were analyzed qualitatively using a modified thematic content analytical approach (e.g., Patton, 1990). One author reviewed all participant audio logs and debriefing interviews while examining relevant maps and photographs. Generalizations extracted from participant commentary were complied during the initial review; generalizations included such items as identification of rocks and features, taking measurements, noting location, explicit indicators of mapping strategy or planning, confidence, interpretation of rock or rock feature relationships, obstacles or problems encountered, and making predictions. These generalizations were next grouped into four themes, and a subset of audio files were reviewed by a second author to ensure overarching agreement with generalizations and themes. 

We took this condensed approach to interrater reliability primarily because only three participants completed audio logs and interviews; this small sample size precludes generalization of our findings, regardless of steps taken to establish reliability. Finally, while some themes emerged from the data naturally, we recognize that our prior knowledge, field experience, and familiarity with expertnovice literature likely influenced these themes. In particular, as fieldtrained geologists we expect mapping activities to involve repeated identification of objects and location of oneself on a base map; it is therefore difficult to ascertain if all of the thematic codes emerged naturally from the data or were established a priori. 

RESULTS 

Quantitative Data: Maps and Tracks - Maps of Lighthouse Point produced by the seven participants were highly variable across the entire population, although six of the maps contained at least some features, such as cross cutting relationships, in common (Figure 1; for full data set see online supplement). Given this disparity, it was nearly impossible to associate a particular lithological unit across participant maps at both study sites. However, it was possible to examine the overall percentage of lithological unit classification in the total mapped area (Table 3), and in particular locations of interest at the site. It was also possible to examine the complexity of the map by examining the number of units and polygons per unit on each map (Figures 1 and 2). 

In general, we found that student mappers (N3, N4, and N5) had similar overall percentages of lithological units (Table 3). We also found total map area devoted to different units was similar for experts (E6 and E7) (Table 3). The two most inexperienced mappers (N1 and N2) produced maps with the least amount of complexity in terms of number of units and polygons per unit. The geology students (N3, N4, and N5) produced the most complex maps, and the two professionals (E6 and E7) produced slightly less complex maps (Figures 1 and 2).

Crosssections produced at the Harvey Quarry site were even more variable across the entire population (Figure 3; for full data see online supplement); in fact, we were unable to clearly identify common lithologic units across all crosssections, preventing comparison of number of units and polygon locations. We did note that expert E6’s crosssection was less structurally complex than all other participants except novice N1. Expert E7 had the most structurally complex crosssection (Figure 3). 

Analysis of individual participant tracks at both sites indicates that participants mapped at very different speeds from one another, and at very different speeds during various portions of the task. For example, novice N2 finished the Lighthouse Point task in just over one hour, whereas experienced mappers E6 and E7 finished in 3.5 and 4 hours, respectively. The remaining students took 4.5 to 5.5 hours to finish the actual mapping plus other course requirements (e.g., lithologic descriptions and final map). Harvey Quarry data show similar trends.

FIGURE 2. Example Lighthouse Point generalized GPS tracks and maps from student N3 and volunteers N2, E6, and E7. Arrows on the GPS tracks show the direction of travel. Generalized GPS tracks, kernel density maps, and geologic maps for all participants are available in the online supplement.  

FIGURE 3. Example Harvey Quarry crosssections from student N3 and volunteers E6 and E7. “RE” indicates rocks intermittently exposed and mapped along the traverse. “PU” indicates rock units, contacts, and structures projected underground. Total length of the traverse is about 450 m with flags every 30 m, and there is no vertical exaggeration; north is to the right. Crosssections from remaining participants are available in the online supplement.

Qualitative Data: Audio Logs and Interviews -Coding of the audio log and interview transcripts produced by subjects N2, E6, and E7 revealed a dozen generalizations that could be grouped into four major themes: navigation and spatial awareness (“Where am I?”), identification of features (“What am I looking at?”), synthesis and model testing (“What does it all mean?”), and metacognition (“How am I doing?”). 

Navigation and Spatial Awareness: Where am I? Two key aspects of geologic mapping are the ability to orient oneself in space, either in relation to landscape features or to distance markers, and to negotiate between the actual landscape and the map. Participant comments related to both of these aspects were placed in this category. All three participants made frequent references to their location in space relative to obvious natural and manmade features (coastline, buildings, road, obvious changes in topography, and marker flags). All participants also used orienting techniques throughout each task, particularly at the beginning. Orientation strategies included moving to a high point to get an area overview, walking the entire transect, and changing perspective by moving close to and backing away from the outcrop. For example, 

Novice N2: I guess what I am doing now is just trying to take an overview of what the area looks like, get familiar with it since I don’t know anything about it. …just looking at the structures, the topography, and exposures. (LP audio log) 

Although she made frequent reference to her location in space, the inexperienced mapper (N2) had difficulty negotiating between the map and the landscape, and as a result both of her maps were misscaled. She noted these problems in her interview: 

Novice N2: I had a difficult time finding the flags…It kept throwing me off that the actual outcrop was almost a third smaller than the subsurface area.” [note: base map projected subsurface contacts] (HQ interview)

Neither of the experienced mappers verbalized this problem. In fact, both experienced mappers allowed aspects of the landscape, such as the scale of the map area and quality of exposure, to dictate their mapping strategy: 

Expert E7: …I realize that I’m mapping an area that’s a few hundred feet long and a couple hundred, 200 or 300 feet wide, and I wonder should I take a walk about to see what I can see? The types of exposures and where the best exposures are… I think the first thing I might do, as far as improving my efficiency would be to walk up at the high point… and have a look over the map area (LP audio log) 

Although this navigational technique is similar to that used by the novice, this experienced mapper explicitly uses the overall size of the map area to control his strategy. 

Procedural and Declarative Knowledge: What am I looking at? A large portion of the audio log entries referred to identification of rocks and geologic features, and to determining rock and feature orientation. Many common rock types were immediately recognized and named (e.g., basalt, quartzite, slate). When rocks could not be easily identified, participants used additional textural and compositional information, for example: 

Expert E6: I’m going to call this a dolomite; it’s definitely soft enough to be readily scratched with a rock hammer; it does not react to acid; it’s purple. At a glance it looks pretty similar to the quartzite; it’s a densely interlocked granular structure of a pale purple crystalline rock but it’s too soft to be quartzite and it’s got that marly, stromatolitic kind of texture on the weathering surfaces. (HQ audio log) 

Rock features, such as bedding and joint sets, contacts between rock units, and structural features, such as joints and folds, were also commonly noted and mapped. Both expert participants, and to a lesser extent the novice participant, frequently took measurements of rock orientation (strike and dip) as well as estimated thickness of small rock layers. A significant difference between the novice and more experienced mappers was that the experienced mappers often (though not always) recognized rock types and structural features more readily than the novice mapper, who often struggled with rock identification (especially with the identification of weathered rocks): 

Novice N2: … it was hard for me to identify [rocks] in the field because of the Ward’s samples that were given in the lab, which are perfect representations of perfect rocks and minerals. I can honestly say that absolutely none of them … come with algae and weathering and erosion and anything else subject to the environment…[note: Ward’s Scientific is a company that commonly supplies samples for education] (LP Interview) 

Additionally, E7 included many more descriptions of structural features (faults, shear zones) than E6 in his audio log, which correlates with greater complexity in his map (Figure 2). 

Synthesis and Model Testing: What does it all mean? All participants verbalized interpretation of rocks and rock features to some extent; for example, making mention of rock classification (igneous, sedimentary, or metamorphic), making judgments about subdividing a single unit into multiple ones, speculations about the nature and type of rock deformation, and interpretation of rock origin or deformation history. For examples: 

Novice N2: This appears to be an intrusive rock type… Then there’s faulting… This area contains a series of fractures and faults. (HQ interview) 

Expert E6: I still think the granite is intrusive… All of this [granite] is parallel to the two fault planes I measured earlier. So if the granite was synchronous with some kind of regional deformation that might explain the shearing and the intermixing of the rocks here in this transitional zone. (LP audio log) 

The two expert participants were far more likely to verbalize interpretation of map features in the audio logs, and both experienced mappers tested their tentative mental models against additional observations: 

Expert E6: … I thought, well, it was either a syncline or some kind of repeated section because of faulting…I’m not sure that these strata [south side] duplicate these strata [north side]. (HQ interview) 

Expert E7: …I looked around, but the dominant lithology… was… quartzite, so I’m wondering what happened to the slate? …so I speculated that it’s a fault. (HQ interview) 

Metacognition: How am I doing? Metacognition is generally defined as knowledge of one’s own cognitive processes and includes selfawareness and regulation of the learning experience. All three participants made metacognitive comments in both the audio logs and interviews, mainly as they described their mapping strategies. Both the novice mapper and more expert E6 describe a strategy of looking over an area, determining the rock types and structures, and then look at the same area from a different viewpoint to evaluate the initial opinion: 

Novice N2: … [I] just did my best to scan the area and try to find any samples or anything that I could do to help me describe what the rock types were and what was going on. So I would scan broadly and then narrow it down and then if I moved it to another area I would scan again and go narrow. And then … when I was working my way back, I would still look back over the areas that I had already mapped just to make sure my point of view wasn’t going to change my opinion of what I had done. And I did that with each spot that I went to. (LP interview) 

Expert E6: My general game plan here is to just start at one end and approach this pretty linearly, just keep walking to the north, take strike and dips, take lithologic descriptions, and see what I see as I work my way just along the face linearly. (HQ audio log)

Metacognition also includes one’s awareness of distractions to the task at hand. All three subjects pointed out distractions to the task, although overall the novice mapper made far more frequent comments about physical and psychological distractions than the experts. For examples: 

Expert E6: All righty… I lost my acid bottle somewhere. I made it to about [the] 1,000 foot mark there and realized I might have needed it and I’ve got to backtrack now and try to find it. (HQ audio log) 

Expert E7: I walked across the highway, which is the most dangerous thing I’ve done today. (HQ interview) 

Novice N2: The area that was covered with shrubs… was kinda creepy and I stepped on a dead animal… so I didn’t like going through that area. And then this area… was incredibly close to the highway and pretty steep, so I didn’t feel incredibly comfortable at that end. (HQ interview)

DISCUSSION

The mixed methods approach presented here provides useful insights into geocognition when data are triangulated; multiple analyses on some data, such as the maps, are much more insightful than a single analysis would have been. The collection and analysis of GPS tracks offers evidence of actual participant movement during mapping, while ThinkAlouds offer tangible links to cognitive processes underlying behavior. We provide below some preliminary suggestions for collection of each of the data sets utilized here. In addition, we discuss findings about expert-novice strategies as suggested by these data, with the caveats that the participant population was small and nonrandom, treatments were nonequivalent, and thematic codes may or may not have authentically emerged from the data. We follow this brief analysis with recommendations for application of this methodology to more robust geocognitive investigations of experts and novices in authentic field settings. 

Findings: Methods – This study enabled us to test methods of collecting and analyzing a variety of data sets. Based on our experience and preliminary findings emerging from the data, we can make some recommendations for future work of this type. Due to the wide variety of maps and cross-sections produced by the participants, we found that percentage of area mapped as specific lithological units (rock types), total number of mapped units, and average number of mapped polygons per unit provided the most useful analyses. In order to make more meaningful comparisons between participant maps, we recommend that all participants share the same familiarity with the study area and the same expectations for naming of map units. We also propose that giving participants a standard way to indicate certainty on their maps (e.g., solid lines for certain contacts, dashed lines for interpolated contacts) would also enable more meaningful comparisons. Although purposeful samples, such as that used here, are a common mechanism for easily obtaining novice participants, we recommend that all subjects in novice-expert studies be recruited in the same manner.

The ArcGIS analyses of the GPS data provide invaluable insight into participant movement. We found that the automated GPS setting was able to capture finescale movement of the participants through the site; however, the nonequal time increments brought unexpected difficulty in determining appropriate statistics for temporal analyses. We did find there were clear advantages to the automated setting, including a significant reduction in the size of the data set. The automated setting also defined participants’ stops in a uniform and automated way, rendering this aspect of the analysis much easier. Overall the automated setting was preferred over the 3minute GPS record used by Riggs et al., (2009). By itself, the GPS data provide evidence of subject movement; this movement can be fully understood when linked to audio data suggestive of underlying cognitive processes. 

Analyses of ThinkAloud audio logs and followup interviews suggest that these are a powerful means of elucidating cognitive processes in the field for both novices and experts, and provide the cognitive link necessary for fully interpreting navigation data. Our volunteer participants were instructed to simply record their thought processes as they were mapping; all three participants recorded ~2050 total minutes of comments during mapping (total mapping time ranging from just over 1 hour to just under 4 hours), suggesting that many thought processes were captured. For future work, we recommend that audio comments be time stamped, so that portions of the audio log can be exactly and easily correlated with GPS tracks. 

In addition to the ThinkAloud audio logs, followup interviews proved important for clarifying maps and strategies. For example, we were able to ask participants to clarify their map symbols. We were also able to explicitly ask about participant certainty related to the map produced and about particular strategies used to create the map. We recommend that in future work, all subjects participate in a followup interview immediately following the mapping experience. 

In general, we noted good agreement with participant recollections of the map experience as expressed in the interviews and in the actual real-time audio logs. Overall, the audio logs captured many fine-scale thought processes that were not evident in the interviews. For example, expert E6 mentions in the interview that he worked out the crosscutting rock relationships at the Lighthouse site by going to two critical locations. Yet the audio log reveals exactly how he worked out the age relationships of the rocks, the clues he used, and that he revised his opinion several times before drawing a final conclusion. These fine scale thought processes would not become apparent from map, track, and interview data alone. Given the effort required to collect, transcribe and analyze audio log data (estimated at 1020 hours per audio log), we recommend that the specific research questions of importance to the researcher should dictate whether interviews alone (which capture generalized thinking) or interviews coupled with audio logs (which capture moment-by-moment thinking) would be most appropriate and necessary for probing thinking in the field. 

Findings: GPS Tracks and Maps - At the Lighthouse Point site, the analysis of map complexity (number of units and polygons per unit) revealed that complexity was the greatest for those participants we identified as intermediate along the expert-novice continuum (i.e., the geology students had the most complex maps; Figures 1 and 2). We also found a high degree of similarity between the proportion of units mapped as particular lithologies (Table 3). The field course students (N1, N3, N4, and N5) were taken to several sites in the study area and shown rock types and crosscutting relationships by their instructor. We therefore speculate that these differences may be due to the information received by students from the course instructor, rather than necessarily an expert-novice difference. However, comparison between the remaining participant maps (N2, E6 and E7) reveals some interesting expert-novice differences. At Harvey Quarry, expert E6’s cross-section was less structurally complex than expert E7’s cross-section (Figure 3). We speculate that E7’s prior mapping experiences may have made him more aware of structural features, as well as more likely to interpret rock features as structures than E6. 

Our analysis of the GPS data suggest that collecting waypoints under the automatic setting captures fine-scale movements of the wearers, enabling a more accurate log of participant locations to be captured and facilitating some analyses. This is probably similar to collection of time-differentiated data at much shorter time intervals (tens of seconds) than the three minutes intervals suggested by Riggs et al. (2009); this suggestion will be tested in future work. 

Kernel density analysis of GPS tracks (Figure 1) indicates that certain portions of the landscape were likely to be covered multiple times by a single participant, while relatively few participants attempted to examine those areas that were difficult to access. Analysis revealed several broad categories of participant movement; for example, some participants spent a relatively long time on the catwalk crossing the center of Lighthouse Point, while other participants spent the majority of their time walking the outside edges of the landscape. Some participants mapped quickly “on the fly” and then sat for long periods to draw maps and compile notes, whereas others drew and described as they walked through the area. Additionally, some participants utilized a particular strategy (example: walk perimeter completely twice, revisit locations of complex relationships), while other participants walked much further distances, backtracking several times in large loops and seemingly at random. Overall, kernel density analysis suggests that experts (E6 and E7) display economy of movement during mapping, as suggested by simpler movement tracks and less backtracking than novices. These two experienced mappers also spent more time in key areas, such as near contacts, than novices. These map and track data suggest that underlying differences in participant experience, background, and mapping strategies will influence field behavior and interpretations. 

Findings: Audio Logs and Interviews - The ability to produce a geologic map requires interpretation of the rocks and structures that are present in the map area; in other words, to synthesize the rock and rock feature data into a model that consistently explains observations. Interestingly, we found no evidence that the novice participant synthesized observations into a model, correct or incorrect, of Harvey Quarry (a syncline) on either the Audio Log or during interview probing designed to encourage synthesis. For example: 

Interviewer: Did you see anything that might be called a structure? Novice N2: This appears to be an intrusive rock type… Then there’s faulting… This area contains a series of fractures and faults… it seems to me that these was some serious uplift… but I don’t know if I can generally say there’s one thing going on. (HQ interview) 

Conversely, the two expert participants were far more likely to verbalize interpretation of map features in the audio logs; additionally, both mapped the quarry site as a syncline and were able to explain how they reached this interpretation. In addition to synthesizing data and forming mental models that explain the observations, both experienced mappers tested their tentative mental models against additional observations. Interestingly, each expert had a different modeltesting strategy. Expert E6 spent a significant amount of time understanding contact relationships and how rock units fit together; he made initial observations followed by later interpretations, and although he found additional data that did not fit his model, it was insufficient for him to immediately change his model (Figure 2): 

Expert E6: …it was only later, after I made it all the way to the north end and started working my way back that I started to realize all the strikes and dips were pointing toward the center, and I thought, well, it was either a syncline or some kind of repeated section because of faulting. So basically descriptive in the beginning and interpretive toward the end. In general I drew a syncline. Not 100% positive that that’s accurate, because there’s a lot of cover and this is all weathered really recessively in here, and I’m not sure that these strata [south side] duplicate these strata [north side]. So this might be better explained as a structure, like a fault instead of a fold. (HQ interview) 

Expert E7 utilized a different modelbased strategy, verbalizing interpretations of the origin of rocks and structural features during mapping, as well as attempts to infer the geologic history of the site. He also fit together observations into models, but in contrast to E6 he made initial models to test and often changed his model based on new evidence (Figure 2): 

Expert E7: Continuing back into the woods, there was a beautiful outcrop of stromatolites that are essentially horizontal… so I was speculating that I was in the middle part of a syncline… As I came out of that… I was confused because if this was truly a syncline, then I should come out of the basal part of the stromatolite and back into the slate. I looked around, but the dominant lithology… was… quartzite, so I’m wondering what happened to the slate? …so I speculated that it’s a fault. (HQ interview) 

When E6 expressed confusion about the rock types, location of contacts, and his interpretation of the rock structure (the syncline), he revised his strategy: 

Expert E6: I’m going to change my strategy a little bit here. I’m going to walk all the way to the northern end and start working my way back. The middle zone here, where it’s weathered recessively and heavily vegetated, I’m going to come back to; I’m going to save for last. So I’m going to, from here just walk to the far north end and then start backtracking my way towards the center. (HQ audio log) 

All three participants made frequent comments expressing degrees of certainty about their location in the map site, rock and rock structure identifications, and interpretations of the rocks and structures (their mental models). Comments ranged from being “sure” of a particular location, rock type or feature (e.g., measuring a bedding plane), to whether an observation “makes sense” in light of other observations, to “just guessing” about a rock type or conclusion. All of the participants could point out areas of their maps that they felt certain and uncertain about, and could articulate reasons for their degree of certainty. However, each participant was unsure for different reasons. The inexperienced mapper (N2) expressed the most uncertainty over rock type identification. Participants E6 and E7 both felt confident of the rock types they mapped, but were unsure of contacts and structural features; for example, participant E6 wondered about the overall syncline structure he had mapped at the HQ site, and participant E7 felt unsure of his identification of several faults at that same site.

Overall, key differences between novice and expert mapping were apparent, and these resulted in both bedrock maps and GPS tracks that varied in complexity. Based on interpretations of the audio log data, the novice mapper was far more likely than the two experts to struggle with determining her location, negotiating between the map and the landscape, and identifying rocks and rock features. She also noted many more distractions to the mapping task. More critically, the novice mapper did not synthesize data to form a mental model of the rock relationships at the mapping site, nor did she test her model against additional observations. We recognize that one subject cannot adequately “represent” all novice mappers. However, anecdotal evidence, based on personal experience teaching a field methods course (author HP) and conversations with colleagues who teach field mapping, suggests that the difficulties described by the novice mapper (rock identification, location, synthesis of data) are typical of students new to mapping. 

Although the two experts had different approaches to developing and testing models, both could clearly articulate the strengths and weaknesses of their models and, consequently, their final map. Additionally, both experts made frequent reference to the certainty of their observations and interpretations, and both adjusted their mapping strategies to account for new observations or their own new insights. We conclude, therefore, that metacognitive aspects (e.g., planning strategy, monitoring performance, revising strategy, staying on task, and evaluating the final product) play a perhaps previously unrecognized but vital role in field mapping performance. 

We speculate that the experts’ ability to monitor and evaluate their performance contributed to how they planned their movement through the map area to take advantage of better rock exposures and recognize key features. In addition, the GPS tracks and complexity analysis, coupled with audio logs, suggests that mapping strategy may impact map complexity (similar to results reported by Riggs et al., 2009). Given the mixed volunteer and purposeful sample used in this study, we chose not to evaluate participant maps for correctness relative to a published map or expert-provided answer key. In the future, comparison of maps generated under equal treatment conditions to a published map would be advantageous. In addition, comparison of map correctness and ThinkAloud data, as cognitive proxies, to GPS tracks, as navigational proxies, will generate a deeper understanding of the influence of cognition on behavior, and vice versa, in field settings. Overall, these findings agree with the limited existing work, such as that of Brodaric and Gahegan (2001), suggesting that geocognition in field settings is situated, and that cognitive processes are tied to both environmental inputs and prior knowledge.

IMPLICATIONS AND RECOMMENDATIONS

Results of this study suggest that scientists are quite varied in how they perceive and process data, and that geologic data are perhaps more varied by perception of the individual scientists than we like to believe. For example, at the quarry site, expert E6 was more inclined to treat lithological boundaries as contact relationships whereas expert E7 was more inclined to treat them as structural discontinuities. Although both experts mapped a syncline, the two maps differed significantly in structural complexity as a result of these personal biases. Although we would like to think of geological data, and in particular geological maps, as objective representations of the surface and subsurface, they are perhaps more laden with personal interpretation than we care to admit. 

Our results also suggest that in addition to teaching the mechanics of field mapping (i.e., note location, identify rocks and features, take measurements), field mapping instruction should focus on the metacognitive aspects of learning as well. Explicitly teaching metacognitive aspects of problemsolving in physics, for example, has been shown to improve students’ problemsolving skills as well as their conceptual knowledge of physics (e.g., Van Heuvelen 1991). In particular, geology students may benefit from explicit instruction in how to take observations and synthesize these into models, and how to test the models against additional observations. Explicit instruction in planning and self-monitoring could be beneficial as well; for example, one might teach students how to use aspects of topography in planning a mapping route (also suggested by Riggs et al., 2009). Actual, physical modeling of the metacognitive aspects of mapping practice may help students understand these key components of successful mapping. 

Future work will replicate this study on a larger scale with participants representing many levels of expertise, including undergraduate students on their first mapping experience, advanced students who have completed field mapping courses, and professional geoscientists with a range of mapping experience. We will look to support tentative conclusions described above; while this study provided a rich source of information on geologic mapping strategies and behaviors, we do not yet have enough data to distinguish between patterns indicative of expertise in mapping and patterns idiosyncratic to individual participants. We also plan to investigate how mapping performance varies with expertise in different areas of geology, in order to further determine how expertise “colors” how geoscientists perceive and interpret rocks. Finally, we expect that this line of research should lead to a better understanding of cognitive processes needed for effective field mapping, and ultimately will point us towards mechanisms for engendering these processes in students.

Acknowledgments

We would like to thank our study subjects for their time and willingness to participate in this research. We thank Whitney Alrick and Magdalena Wisniewska for their assistance with GIS analysis. Conversations with Zach Hambrick helped to clarify our thinking about cognitive themes in the data. We thank colleagues Charles Henderson, Herb Fynewever, David Rudge, and Joseph Stoltman at WMU for feedback that helped to improve this manuscript. Reviews by Kristen St. John, Roger Trend, and two anonymous reviewers substantially improved this manuscript. This research was funded in part by a WMU Faculty Research and Creativity Activities Fund (FRACASF) grant to H. Petcovic. Ongoing collaboration between authors is supported by the National Science Foundation under Grant No. DRL0815764. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

An online supplement to this article, with additional figures and images, is available at http://www.journalofgeoscienceeducation.org/

REFERENCES

1. Abel, M., Mastella, L.S., Lima Silva, L.A., Campbell, J.A., and De Ros, L.F., 2004, How to model visual knowledge: a study of expertise in oil-reservoir evaluation, Lecture Notes in Computer Science: Berlin, Springer, p. 455-464.
2. Anderson, K.C., and Leinhardt, G., 2002, Maps as representations: Comparison of expert/novice projection understanding: Cognition and Instruction, v. 20, p. 283-321.
3. Barkowsky, T., and Freksa, C., 1997, Cognitive requirements on making and interpreting maps: Lecture Notes in Computer Science, v. 1329, p. 347-361.
4. Bond, C.E., Gibbs, A.D., Shipton, Z.K., and Jones, S., 2007, What do you think this is? “Conceptual uncertainty” in geoscience interpretation: GSA Today, v. 17, p. 4-10.
5. Bransford, J.D., Brown, A.L., and Cocking, R.R., 2000, How People Learn: Brain, Mind, Experience, and School: Washington, D.C. , National Academy Press.
6. Brodaric, B., and Gahegan, M., 2001, Learning geoscience categories in situ: Implications for geographic knowledge representation: Proceedings GIS ’01, p. 130-135.
7. Chi, M.T.H., 2006, Two approaches to the study of experts’ characteristics, in Ericsson, K.A., N., C., Feltovich, P.J., and Hoffman, R.R., eds., The Cambridge Handbook of Expert Performance: New York, Cambridge University Press, p. 21-30.
8. Chi, M.T.H., Feltovich, P., and Glaser, R., 1981, Categorization and representation of physics problems by experts and novices: Cognitive Science, v. 5, p. 121-152.
9. Creswell, J.W., 2009, Research Design: Qualitative, Quantitative, and Mixed Method Approaches: Thousand Oaks, CA, Sage Publications, Inc.
10. Donovan, S.M., and Bransford, J.D., 2005, How Students Learn History, Mathematics, and Science in the Classroom: Washington, D.C., National Academies Press.
11. Eley, M.G., 1991, Selective encoding in the interpretation of topographic maps,: Applied Cognitive Psychology, v. 5, p. 403-422.
12. Ernst, W.G., 2006, Geologic mapping – where the rubber meets the road, in Manduca, C.A., and Mogk, D.W., eds., Earth and Mind: How Geologists Think and Learn about the Earth, Volume 413: Geological Society of America Special Paper, Geological Society of America, p. 13-28.
13. Feltovich, P., Ford, K., and R., H., 1997, Expertise in context: Cambridge, MA, MIT Press.
14. Frodeman, R., 1995, Geological Reasoning: Geology as an Interpretive and Historical Science: Geological Society of America Bulletin, v. 107, p. 960-968.
15. Hmelo-Silver, C.E., and Nagarajan, A., 2002, "It's harder than we thought it would be": A comparative case study of expert-novice experimentation strategies: Science Education, v. 86, p. 219-243.
16. Hoffman, R.R., Shadbolt, N., Burton, A.M., and Klein, G.A., 1995, Eliciting knowledge from experts: a methodological analysis: Organizational Behavior and Human Decision Processes, v. 62, p. 129-158.
17. Hoz, R., Bowman, D., and Kozminsky, E., 2001, The differential effects of prior knowledge on learning: A study of two consecutive courses in earth sciences: Instructional Science, v. 29, p. 187-211.
18. Kastens, K.A., Agrawal, S., and Liben, L., 2009, How students and field geologists reason in integrating spatial observations from outcrops to visualize a 3-D geological structure: International Journal of Science Education, v. 31, p. 365-394.
19. Kastens, K.A., and Ishikawa, T., 2006, Spatial thinking in the geosciences and cognitive sciences: A cross-disciplinary look at the intersection of the two fields, in Manduca, C.A., and Mogk, D.W., eds., Earth and Mind: How Geologists Think and Learn about the Earth, Volume 413: Geological Society of America Special Paper, Geological Society of America, p. 53-76.
20. Kuhn, W., 2001, Ontologies in support of activities in geographical space: International Journal of Geographical Information Science, v. 15, p. 613-631.
21. Libarkin, J.C., 2006, Geoscience Education in the United States: Planet, v. 17, p. 60-63.
22. Libarkin, J.C., and Anderson, S.W., 2007, Development of the Geoscience Concept Inventory, National STEM Assessment Conference: Washington, D.C., p. 148-158.
23. Lobben, A.K., 2004, Tasks, strategies, and cognitive processes associated with navigational map reading: A review perspective: The Professional Geographer, v. 56, p. 270-281.
24. Macdonald, R.H., Manduca, C.A., Mogk, D.W., and Tewksbury, B.J., 2005, Teaching Methods in Undergraduate Geoscience Courses: Results of the 2004 On the Cutting Edge Survey of U.S. Faculty Journal of Geoscience Education, v. 53, p. 215-219.
25. Manduca, C.A., and McDaris, J., 2007, The NAGT teaching in the field website: Sharing pedagogy and resources: Geological Society of America Abstracts with Programs, v. 39, p. 544.
26. Parunak, H.V.D., Brueckner, S.A., Matthews, R., and Sauter, J., 2006, Swarming methods for geospatial reasoning: International Journal of Geographical Information Science, v. 20, p. 945-964.
27. Patel, V.L., Kaufman, D.R., and Arocha, D.F., 2002, Emerging paradigms of cognition in medical decision-making: Journal of Biomedical Informatics, v. 35, p. 52-75.
28. Patton, M.Q., 1990, Qualitative evaluation and research methods: Newbury Park, Sage 523 p.
29. Petcovic, H.L., and Libarkin, J.C., 2007, Research in Science Education: The Expert-Novice Continuum: Jounal of Geoscience Education, v. 55, p. 333-339.
30. Pick, H.L., Heinrichs, M.R., Montello, D.R., Smith, K., Sullivan, C.N., and Thompson, W.B., 1995, Topographic map reading, in Hancock, P.A., Flach, J.M., Caird, J., and Vicente, K.J., eds., Local Application of the Ecological Approach to Human- Machine Systems: New York, Erlbaum, p. 255-284.
31. Raab, T., and Frodeman, R., 2002, What is it like to be a geologist? A phenomenology of geology and its epistemological implications: Philosophy & Geography, v. 5, p. 69-81.
32. Raubel, M., 2001, Ontology and epistemology for agent-based wayfinding simulation: International Journal of Geographical Information Science,, v. 15, p. 653-665.
33. Riggs, E.M., Lieder, C.C., and Balliet, R.N., 2009, Geologic problem solving in the field: Analysis of field navigation and mapping by advanced undergraduates: Journal of Geoscience Education, v. 57, p. 48-63.
34. Sabers, D.S., Cushing, K.S., and Berliner, D.C., 1991, Differences among teachers in a task characterized by simultaneity, multidimensionality, and immediacy: American Educational Research Journal, v. 28, p. 63-88.
35. Schlieder, C., 2005, Representing the meaning of spatial behavior by spatially grounded intentional systems, in Rodriguez, M.A., Cruz, I.F., Egenhofer, M.J., and Levashkin, S., eds., Lecture Notes in Computer Science, Volume 3799.
36. Turner, A., and Penn, A., 2002, Encoding natural movement as an agent-based system: an investigation into human pedestrian behaviour in the built environment: Environment and Planning B: Planning and Design, v. 29, p. 473-490.
37. Van Heuvelen, A., 1991, Learning to think like a physicist: a review of research-based instructional strategies: American Journal of Physics, v. 59, p. 891-897.
38. Wilson, M., 2002, Six views of embodied cognition: Psychonomic Bulletin & Review, v. 9, p. 625-636.
39. Zeilik, M., Schau, C., and Mattern, N., 1999, Conceptual astronomy. II. Replicating conceptual gains, probing attitude changes across three semesters: American Journal of Physics, v. 67, p. 923-972.