CSCE 420 Lecture 24

From Notes
Jump to navigation Jump to search

« previous | Tuesday, April 16, 2013 | next »


Agent Architectures

Decision-making Implementation (Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \pi(s) = a} )

  1. Sense
  2. Decide
  3. Act
  4. repeat...

Similar to other fields'

  1. Observe
  2. Orient
  3. Decide
  4. Act
  5. repeat...

Simple Reflex

  1. sensor input
  2. what does world look like now? → state
  3. what action should I do now? → action
  4. perform action
  • Table-lookup: determine actions in a table based on sensor inputs (doesn't scale well)
  • Rule-based Agent: Implement Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \pi} with set of "condition → action" rules (requires conjunctive antecedents)
    • Rules should be ordered by priority
    • Stop after first match
Figure 1. Ghengis Robot

Genghis Robot (Rob Brooks, MIT) has simple, reactive controllers for legs:

  1. push backward
  2. lift up
  3. move forward
  4. if stuck, retract and lift higher
  5. set down
  6. repeat...

Model-Based

Maintain an internal representation of the world state

  1. sensor input
  2. update internal state
  3. what does the world look like now? → state
  4. what action should I do now? → action
  5. perform actions

Goal-Based

Describe in KB what goals need to be accomplished

  • Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle KB \wedge \text{state} \wedge \text{goals} \models \text{do}(a_i)}
  • Try to infer for all Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle a_i \in \text{Actions}}
  • use Situation Calculus to encode preconditions and effects of actions in Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle KB}


OR use planners to derive sequence of actions (plan) by state-space search (e.g. A*)

In either case, output of actuators follows a plan

Utility-Based

  • Utility Function Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle u(s) \to \mathbb{R}}
  • Transition function Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle T(s,a) \to S}
    • defines outcomes of actions
    • could be probablistic (distribution over successor states)
  • Reward/Cost function Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle R(s,a) \to \mathbb{R}}
    • payoff of action
  • Goal: maximize reward over time: Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \sum_{t=0}^{\infty} \gamma^t \, R_{a_t}(s_t, s_{t+1})} , where Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \gamma \in \left[ 0,1 \right]} is a geometric weighting scalar/constant that helps series to converge.
    • smaller Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \gamma} would look for immediate payoff
    • larger Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \gamma} would be longer term

Plans are encoded in policies (mapping states to actions)

Computing optimal policy Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \pi^*} that maximizes long-term discounted reward

  • evaluation via Bellman equations: Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle V^*(s) = \max_{a \in A(s)} \sum_{s'} {Pr}_{s\,s'}^a \, \left[ \mathcal{R} + \gamma \, V^*(s') \right]}
  • Value function gives utility of each state (which depends on its neigboring states)
  • Therefore values are all Coupled
Retrieved from "https://notes.komputerwiz.net/w/index.php?title=CSCE_420_Lecture_24&oldid=3045"