Using ChatGPT to Read Very Short Introductions

Following my birthday last month, the thought occurred to me that this could be a year of learning about topics I’m curious about by reading through as much of Oxford UP’s Very Short Introductions series as I can. As an informal experiment, while reading the series’ entry on amphibians, I tried asking ChatGPT to give me some questions to answer by the time I finished reading the book. This blog post is a summary of how this experiment went.

Other ways to read Very Short Introductions

Before reading Amphibians: A Very Short Introduction, I had read two other books from the series: the one on intelligence, and the one on bacteria.

While reading Intelligence: A Very Short Introduction several years earlier, I took book notes as I normally do, copying down things I found interesting and things I thought I might want to remember. Looking back, I think I took too many notes.

While reading Bacteria: A Very Short Introduction a couple of weeks ago, every time I finished a chapter, I wrote, from memory, a summary of the information contained within that chapter. This was inspired by a suggestion in the Coursera Learning How to Learn course that you can improve memory encoding after experiencing something you want to remember (at the end of, e.g., a book, a lecture, or a conversation) by writing down, on a scrap of paper, what you remember about the experience. You don’t even need to keep the scrap of paper - it’s the act of recalling from memory and then writing that leads to the benefits.

Why answer questions about a book, anyways?

If you’re trying to learn something from a textbook, one of the best places to begin when starting on a new chapter is at the very end of the chapter, with the questions. First, you do your best to answer them without having read the chapter (while doing your best to remain okay with the fact that you’re getting them wrong). Then, read the chapter, and experience a series of “aha!” moments, which happen every time you come across a piece of information that would have helped you as you were struggling with one of the questions. Ideally, once you finish the chapter, answer the questions again with your newfound knowledge.

When you start by reading the questions at the end of the chapter, you find out what the authors are hoping you’ll take from the chapter. And by doing your best to answer them, you prime your brain to notice and remember these key bits of information when you encounter them.

While my book on amphibians wasn’t a textbook, I wondered whether I could try a similar approach to help me come away from the book with more knowledge. The plan was to come up with a set of five questions that would guide my attention towards important pieces of knowledge about amphibians and their study. After attempting to answer the questions at the beginning, I would return to them every time I finished a chapter, seeing what I could add to my answers.

The book, however, didn’t have questions at the end of each chapter, which meant I would need to come up with some…

So why ask ChatGPT to generate these questions?

While I’ve become more pessimistic about the effects of widely available Large Language Models on education (not to mention society as a whole) over the past couple of years, I think that generating questions for a learner to respond to is one of their more useful applications. Generating questions about a book like this, in particular, plays to an LLM’s strengths, while sidestepping a number of potential pitfalls for learners: LLMs are good at having a general sense about a wide range of topics (whereas I would likely have a hard time coming up with good questions. I don’t really know the state of humanity’s knowledge about amphibians - that’s why I’m reading the book!). On the flip side, since I’m the one answering the questions, I’m not outsourcing the thinking I need to be doing in order to learn. The risk of being misled by hallucinations is much reduced, since I’m not asking ChatGPT to tell me any facts (And if it generates any nonsensical questions, the only negative effect is that, by the end of the book, I’ll be disappointed at having spent time puzzling over a question that I will know by then was useless).

Before asking ChatGPT for questions, as a control, I figured I should come up with my own set of questions to see whether the LLM was doing anything I couldn’t do myself. My questions, and ChatGPT’s questions are listed below, along with the prompt I used.

The Questions

My Questions

• What are amphibians - what features do all amphibians share, and what distinguishes them from other animals? Where do they fit in the tree of life?
• How diverse are amphibians - how many species are there? genera? families? where, in a global sense, are they found, and in what types of ecosystems?
• What ecological role(s) do amphibians play in ecosystems?
• What specific challenges do they face wrt ecosystem disruption, climate change, etc.?
• How has what we know about amphibians changed over time? how did we come to acquire our current knowledge about amphibians?

ChatGPT’s Questions

My prompt:

I am beginning to read a book about amphibians. While I feel I have a decent amount of background knowledge about the biology of life on earth, I don't know much about amphibians in particular, and I'd like to learn as much as I can. Taken from the table of contents, here are the titles of the chapters the book contains:

        - what is an amphibian?
        - the evolution of amphibians
        - reproduction and life history
        - how amphibians move
        - how amphibians feed
        - the amphibians' world: sense organs and communication
        - amphibians and humans
        - conservation and the future of amphibians

        Based on these chapter headings, please provide five questions I can expect to find the answers to by the time I finish the book - questions for me to keep in mind as I read in order to gain as deep an understanding of amphibians as possible.

ChatGPT’s response:

Here are five deep, guiding questions you can keep in mind as you read your amphibian book—each one aligned with multiple chapter themes to help you synthesize and retain a broad, nuanced understanding:

        1. What defining traits distinguish amphibians from other vertebrates, and how do these traits reflect their evolutionary history and ecological roles?
            → Think about anatomy, physiology (e.g. skin, respiration), and their transitional place between aquatic and terrestrial life.
    
        2. How do amphibians reproduce, and in what ways do their life cycles reflect both ancestral aquatic origins and adaptations to land?
            → This will touch on eggs, metamorphosis, reproductive strategies, and developmental stages.
    
        3. In what ways are amphibians adapted to their environments in terms of movement, feeding, sensory perception, and communication?
            → Look for variety across species and connections between form, function, and habitat.
    
        4. How have amphibians interacted with humans culturally, scientifically, and economically—and what impact have those interactions had on amphibian populations?
            → Reflect on amphibians in myth, medicine, research, and as environmental indicators.
    
        5. What are the primary threats facing amphibians today, and what conservation strategies are being used or proposed to ensure their survival? 
            → This question will help you connect the biological with the political and environmental realities amphibians face.
    

        Keeping these questions in mind will help you move beyond memorizing facts and toward understanding amphibians as a dynamic and vulnerable group within Earth's ecosystems.

For my answers, before and after, see my book notes. (I was actually most of the way through the first chapter by the time I generated these questions, so my questions and my initial answers actually reflect the state of my knowledge after reading almost one chapter of the book)

Takeaways

On the whole, I generally think this is a worthwhile practice, and plan to do it again. A few observations:

For one of my questions (“How has what we know about amphibians changed over time?”), the book didn’t really contain any relevant information. Had I read the table of contents like ChatGPT did, I might have known to expect this. Oops! On the other hand, ChatGPT generated one question that I was initially doubtful about (the “Reflect on amphibians in myth, medicine…” bit), but for which there was actually a decent amount of information in the book. I might be able to pattern-match in the future, learning what sorts of questions to ask. But on the whole, this was the idea anyway: ChatGPT has a better sense of the broad outlines of the study of amphibians than I do, and it was able to come up with questions that I wasn’t.

While I initially came up with my own questions as a control, I now think the practice of writing out one’s own questions is a useful part of the exercise. (This could also be a useful practice at the end of the book, too—one that’s useful for learners that come later. Having a goal of coming up with a series of learning goals / evaluation statements would be an interesting lens through which to read a book, too!)

Ten questions in total, however, was too many questions, and I felt a bit bogged down at times at the end of a chapter while trying to flesh out answers to all ten of them. Perhaps in the future, I’ll come up with 5 questions, ask ChatGPT for 5, and then look through them all and keep the 5 or 6 that seem most promising.