Diplomacy
|
Home > Diplomacy > Cogitations > Pressure
Cogitations on AI in the Game of Diplomacy: Pressure
Here is a brain dump of full details my currently Cogitations about Pressure . (Later Cogitations override earlier ones if contradictory.)
[Originally presented in DipAi post #8024, but pressure (and much of the original GT method) is now superseded in a revised GT method.]
For descriptive and conceptual clarity in my GT method, in future I shall refer to "notional probability" and its symbol "p" as "pressure". I shall also replace my term "effective probability" by "desired probability", and add the term "achieved probability" for the measured value – see below. (I may just say "probability" when unambiguous.)
(Notional probability is rather like a probability – which was my intention when I began to conceive it (see below) – but the current concept is not strictly a kind of probability, so is confusing, especially when the "notional" prefix is omitted for brevity. (A true probability must be in the range 0...1, with the sum of the possibilities being 1. At best, what I now call pressure is a pseudo- or proto-probability, used to derive a true probability, as is my concept that I call "intensity". Whereas "unnormalized probability" easily slips mentally to and from a true probability by a simple scaling, and any simple smooth monotonic function could slip with a bit more effort, it is not generally so easy with pressure or intensity. The mapping from pressure to probability cannot be reversed in general, due to the important discontinuity at zero (due to zeroing negative values). The mapping between intensity and probability can only be determined empirically for each operation in a given set of operations, and may be very non-linear, especially for very difficult-to-place operations.) (See Hofstadter's excellent book "Fluid Concepts and Creative Analogies" for "concept slipping", and below.))
So to recap, and expand a bit, using my new terminology....
Monte Carlo simulations of each model of each power are used to converge their estimated best pressures, p, (which may have any real value) for each operation owned by the power being modelled, towards the ideal value, as indicated by the utility, u, perceived by that model, as I described in my recent tome.
When an operation is selected for inclusion in a modified full order-set of a model, it is chosen with probability proportional to its intensity (absolute value being that divided by the current sum for available operations), where, for operation i,
intensity[i] = max(0, pressure[i]+ boost[i] + gaussian_random(temperature[i]))
(Its not physics! But near enough if you use "natural" units to eliminate unnecessary scaling factors.) Boost is gradually adjusted (by an empirical method, to be decided) to tend to minimize the mean square difference between moving means of desired and achieved probabilities, where
desired_probability[i] = max(0, pressure[i])/sum_of_desired_probability_of_operations_of_owner
NB: Boost is added to compensate for systematic differences in the difficulty of placing different operations with average ensembles of other selected operations (some of which might tend to be very incompatible with others), rather than to try to achieve exactly the desired proportions. However, the mechanism should, as a bonus, further improve accuracy while only a small sample size – provided the boost adjustment is gradual enough to avoid plausible pathologically large and/or regular oscillations (phase-locked loop), which could overpower the feedback from utility.
-------------------
Poetically/intuitively, but illogically, pressure is proportional to the probability I would like to use, even if negative – if that were meaningful! For example, if reducing my probability of approaching a certain power seems to correlate with increased utility (with no obvious optimum or even slowdown as probability approaches zero), intuitively, I want to continue to reduce the probability as much as possible – even to be negative if it were meaningful/possible! But, heuristically at least, we can plausibly guess that we could interpret low probability of approach as equivalent to high probability of retreat (at least roughly, if hold does not seem likely to be an optimum). Although we cannot meaningfully use negative probability when selecting an approach operation, we may be able to find one or more operations that tend to move in the reverse direction where we could increase probability. It might be a useful heuristic to increase their pressure (and hence desired probability and intensity), even in the before we have significant (or any) test results for them.
More generally, discovering that ideal pressure for an operation is negative ought to increase pressure for operations that are "opposite" in some sense. This need not be in physical direction of movement;for example, it might be changing a (more abstract) threat to positive cooperation, or building fleets rather than armies. There might be several plausible interpretations for "opposite", and maybe several plausible examples of each, with differing degrees of (metaphorical) alignment. So the negative pressure ought to be re-distributed accordingly (albeit empirically)- in the opposite "direction" in opposite operations.
Significant negative pressures should not occur anyway, however, because that should tend to reduce their sample rate, thereby reducing rate of decrease – eventually ceasing, save for when occasional extreme thermal noise drives intensity positive (which is important, just in case our judgement of negative pressure (dominated operation) was premature). But low positive pressure and desired probability would be little sampled either, so equally here there may be heuristic value in increasing the pressure (hence, intensity and desired probability) of their opposite operations (operations with some properties that have values with opposite sign).
Analogously to the concept of opposites, we can go on to realize that, having discovered that a pressure ought to be changed for one operation, it is likely to need changing in the same direction in similar operations.
In a more advanced implementation, perhaps abstract properties (such as used for opposite/similar) should (also or instead) be directly subject to my method, with their own pressures, intensity and probabilities, and so on. Their desired probabilities would then be realised by updating those of concrete operations, according to how well they match the desired properties.
But enough speculation on the above for now – I am trying to design something concrete that I understand how to implement! Sophisticated, or even simple, analyses of similar and/or opposite operations might be too difficult or expensive, but making best use of hard-won empirical data should at least be considered in due course.
-----------------
I find it interesting trying to introspect how my thought processes operate, as guidance to implementing AI methods. Whether or not the above "opposites" intuition is of value, it is interesting how it developed naturally as I tried to make a useful interpretation of negative pressure. (Just chopping negative values seems bad – at the very least it seemed to be wastefully discarding hard-won data, except for a small effect on chance of producing positive intensity. I was not quite so unhappy when I realised it implied a dominated operation, which I needed to be able to find and eliminate anyway, and this should do the job nicely, with no extra mechanism or processing, with the fast and firm cut-off that I had long sought. Nevertheless, the plausible potential of incorporating conceptually-nearby opposites eventually became apparent too – probably helped by my change of name of the concept! Although a name-change may seem pedantic or pathetic footling (a rose by any other name....), I find that le mot juste helps me focus better on my intended concept, which means strong access to its halos of connotations (which I may wish to metaphorically bend or slip through to obtain a nearby concept).
Regrettably, I had cooled the metaphorical temperature of the pressure concept and its name too fast! Fortunately, some metaphorical thermal noise today prompted me to reheat and re-anneal the metaphorical region a little. :-) How I would love to program an AI to do such things at such abstract levels!