Python Chatbot Using NLP Project Flashcards

24 sample cards from the Chatbot Using NLP Project deck

Sampled from the end of the deck, so these are different cards from the ones shown on the syllabus page.

1. Define epoch, batch size, and iteration in model training.

   An epoch is one full pass over the training data. Batch size is the number of samples processed before a weight update. An iteration is one weight update; iterations per epoch equal dataset size divided by batch size.

2. What is the learning rate and what happens if it is too high or too low?

   The learning rate $\eta$ scales each weight update. Too high causes overshooting/divergence and unstable loss; too low causes very slow convergence and risk of getting stuck in poor minima.

3. What is overfitting and name two techniques to reduce it.

   Overfitting is when a model memorizes training data and generalizes poorly to new data (low train loss, high validation loss). Mitigations: dropout, regularization (L1/L2), early stopping, more data, or data augmentation.

4. Why split data into training, validation, and test sets?

   Training data fits the model, validation data tunes hyperparameters and monitors overfitting during training, and the held-out test set gives an unbiased estimate of final real-world performance.

5. Define accuracy as an evaluation metric and write its formula.

   Accuracy is the fraction of correct predictions: $$\text{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN}$$ It can be misleading on imbalanced intent datasets.

6. Write the formulas for precision and recall and explain what each captures.

   $$\text{Precision} = \frac{TP}{TP + FP}, \qquad \text{Recall} = \frac{TP}{TP + FN}$$ Precision measures correctness of positive predictions; recall measures coverage of actual positives.

7. What is the F1 score and when is it preferred over accuracy?

   F1 is the harmonic mean of precision and recall: $$F1 = 2 \cdot \frac{\text{precision} \cdot \text{recall}}{\text{precision} + \text{recall}}$$ It is preferred on imbalanced data where accuracy alone is misleading.

8. What information does a confusion matrix provide when evaluating a chatbot's intent classifier?

   It tabulates predicted versus actual intents, showing per-class true positives, false positives, and false negatives, so you can see exactly which intents are being confused with one another.

9. What is conversation flow design in a chatbot, and what is a common tool to represent it?

   It is the planned sequence of states/turns a conversation can take, including branches and fallbacks. It is commonly represented with a dialogue/flow diagram or a finite state machine of dialogue states.

10. What is a 'fallback' or default response and why is it essential in conversation design?

    It is the reply used when the chatbot cannot confidently match an intent (low confidence). It prevents dead ends, gracefully handles unknown input, and can prompt the user to rephrase.

11. Contrast rule-based and retrieval/generative response generation in chatbots.

    Rule-based uses predefined patterns and fixed responses (predictable, limited). Retrieval picks the best response from a fixed set; generative models produce novel text token-by-token (flexible but harder to control).

12. After classifying an intent, how does a simple retrieval-based chatbot pick its reply?

    It looks up the predicted intent's tag in the responses store and selects a response (often randomly) from the list of predefined responses associated with that intent.

13. What is a confidence threshold in response generation and how is it used?

    It is a minimum probability the top predicted intent must exceed to be acted on. If the softmax confidence is below the threshold, the bot triggers a fallback instead of risking a wrong response.

14. What is the role of context/state tracking in multi-turn chatbot conversations?

    It stores prior turns and slot values so the bot can resolve references and maintain coherence across turns, enabling follow-up questions and multi-step tasks rather than treating each message independently.

15. Name two common user interfaces a chatbot can be integrated with.

    Examples include a web chat widget (HTML/JS frontend), messaging platforms (Slack, Telegram, WhatsApp, Facebook Messenger), command-line interfaces, or voice assistants.

16. In a typical web chatbot architecture, what is the role of an API endpoint between UI and model?

    The UI sends the user's message via HTTP (e.g. a POST request) to a backend API endpoint; the server runs preprocessing and the model, then returns the generated response as JSON for the UI to display.

17. Why is a lightweight web framework like Flask or FastAPI commonly used to serve a chatbot model?

    It exposes the model as an HTTP API with minimal code, handling routing and request/response so frontends can interact with the chatbot over the network and the model loads once and serves many requests.

18. What should chatbot testing cover beyond just correct intent classification?

    It should test response relevance, handling of unknown/out-of-scope input, multi-turn context, edge cases and typos, fallback behavior, latency/performance, and the end-to-end UI integration.

19. Differentiate unit testing and end-to-end (integration) testing for a chatbot.

    Unit testing checks individual components in isolation (e.g. the tokenizer or intent predictor). End-to-end testing exercises the full pipeline from user input through model to displayed response, as a real user would.

20. What factors influence the choice of a deployment platform for a chatbot?

    Consider scalability, cost, latency/region, ease of deployment, supported runtimes, integration with the target channel, security/compliance, and whether it offers managed scaling (cloud like AWS/GCP/Azure/Heroku) versus self-hosting.

21. What is containerization (e.g. Docker) and why is it useful for deploying a chatbot?

    Containerization packages the app with all its dependencies and runtime into a portable image, ensuring it runs identically across environments and simplifying deployment, scaling, and rollback.

22. Why should model files and dependencies be bundled or referenced explicitly when deploying?

    The production server must load the exact trained model weights and the same library versions used in training; mismatched versions or missing model files cause errors or degraded/incorrect predictions.

23. What does monitoring a deployed chatbot involve, and name two key signals to track.

    It involves continuously observing the live system's health and quality. Key signals: response latency/errors, user satisfaction or fallback rate, intent confidence distribution, traffic volume, and unrecognized (out-of-scope) messages.

24. Why is periodic retraining part of maintaining a production chatbot?

    User language, topics, and needs drift over time (data/concept drift). Collecting new conversation logs and retraining keeps intent coverage current and accuracy high, addressing previously unhandled queries.

What this deck covers

Chatbot Using NLP Project flashcards FAQ