Constructing and clearing combinatorial exchanges using preference elicitation

Constructing and Clearing Combinatorial Exchanges Using Preference Elicitation Trey Smith and Tuomas Sandholm and Reid SimmonsCarnegie Mellon University5000Forbes AvenuePittsburgh,PA15213trey,sandholm,reids@AbstractCombinatorial exchanges arise naturally in multi-agent sys-tems that execute hierarchically decomposed tasks,when theagents have uncertainty about each other’s tasks and eachother’s capability of handling tasks.In a combinatorial ex-change,the information is aggregated to one party who thendecides the task allocation among the agents.Unfortunately,such exchanges can require that bidders calculate and com-municate an exponential number of bids,each of which mayinvolve solving a hard planning problem.We present a designfor an auctioneer agent that can construct and clear a combi-natorial exchange using preference elicitation.This designextends existing analyses of elicitation in the combinatorialauction to the combinatorial exchange.We also introduce theconcept of item discovery that uses elicitation to construct theexchange when there is uncertainty about which items shouldbe considered in the market.Our experimental results,in amulti-robot exploration domain,show that elicitation signif-icantly reduces the number of bids that must be evaluated inorder to clear the market.More important,the proportion ofbids that must be evaluated decreases as we scale to largerproblem instances.We also present experimental results foran anytime version of the elicitation algorithm.IntroductionCombinatorial exchanges are markets with the property thata single bid can jointly express both supply and demandfor combinations of binatorial exchanges arisenaturally in multi-agent systems that execute hierarchicallydecomposed tasks,when the agents have uncertainty abouteach other’s tasks and each other’s capability of handlingtasks.In a market framework,an agent can express its abil-ity to lead a team by making a bid that supplies a high-leveltask,indicates the cost,and demands help in the form of sub-tasks from other robots.For example,a science rover witha coring drill could make a bid that supplies a core sam-ple of a rock and demands an assisting rover to help it alignthe drill properly.By using a combinatorial exchange,wecapture the important property that supply and demand areinter-dependent,which ensures the leader does not win thecontract to supply the high-level task while losing a contractfor one of the necessary sub-tasks.We have applied the proposed auctioneer design to a com-binatorial exchange that models a multi-robot exploration task.Our results show that,for the domain in question,pref-erence elicitation significantly reduces the number of bids that must be evaluated in order to clear the market.More important,the proportion of bids that must be evaluated de-creases as we scale to larger problem instances.We also present experimental results for an anytime version of the elicitation algorithm.Related WorkThis work extends preference elicitation in the combinato-rial auction(Conen&Sandholm2001)to the combinatorial exchange.(Walsh&Wellman1999)examines the formation of demand-driven supply chains in a progressive distributed auction framework.That work inspired our novel concept of dynamic item discovery.Other research has modeled supply chains using distributed auctions(Babaioff&Nisan 2001)and combinatorial exchanges(Walsh,Wellman,& Ygge2000).Many researchers have designed algorithms to efficiently clear the combinatorial auction and generalizations,includ-ing exchanges(Sandholm et al.2001a;Fujishima,Leyton-Brown,&Shoham1999;Sandholm2002).These algo-rithms rely on complete information(all bids known to the auctioneer up front).Our algorithm for clearing the market with incremental bidding repeatedly solves instances of the complete information clearing problem,so these existing al-gorithms can be used to great advantage as subroutines. The application of market-based techniques to multi-robot exploration problems is a relatively new concept.In (Dias&Stentz2000),multiple simultaneous exchanges con-cerning single items were used to distribute targets between a group of exploring robots.Iterative combinatorial market techniques(Wurman& Wellman2000;Parkes1999;Parkes&Ungar2000)can be viewed as another preference elicitation architecture,differ-ing from our current work in their assumptions about query types and how auctioneer and bidder communication are in-terleaved.Some of their concepts,for instance using primal-dual techniques for query selection,may be applicable to our architecture.Problem FormulationIn a combinatorial exchange,the auctioneer receives bids from both buyers and sellers concerning a set of indivisi-ble,distinguishable items.Each subset of the set of items is called a bundle1.A bid consists of a supply bundle,a demand bundle,and a reward.The semantics are that if a bid is accepted,the bidder will supply the items in,consume the items in,and have a reward(increasedhappiness)of.There are submits two kinds of bids:goal bids and pro-duction bids.Goal bids are used to express top-level goalsWe make this important limitation on full expressibility in order to keep elicitation simple and tractable.We refer to the unique demand and reward corresponding to each supply bundle as and.2.Free disposal:If a bidder supplies more items than areneeded,it can throw away the extras at no cost.Thus supplying a smaller bundle is never harder than supplying a larger bundle.If we have bids andfor two supply bundles(the first supply bundle is smaller),then,correspondingly, (a)thefirst demand bundle is smaller()and(b)thefirst cost is smaller,equivalent to a higher reward().For eliciting preferences,the auctioneer has the ability to make two types of queries:1.Value query:ask a bidder for its unique production bidthat contains a given supply bundle.Our main emphasis is on reducing the number of value queries needed to solve the clearing problem.2.Type query:ask a bidder whether it is of the right type to produce item,and if so,what combination group belongs to.We assume that type queries are trivial to an-swer.The auctioneer starts by knowing only the goal bids and the items they demand,which we view as the consumer end of a supply chain.All other items are discovered in the pro-cess of making value queries.When the auctioneer makes a value query for a supply bundle,there may be unfa-miliar items in.In that case,the auctioneer makes a type query to each agent,concerning each unfamiliar item. Those agents capable of supplying a new item may sub-sequently be queried about supply bundles containing,dis-covering further new items that are needed in order to supply .In this way,successively earlier stages of the supply chain are revealed,until we reach bids whose demand bundles are empty.Because item discovery is initiated by value queries,and the market maker is generally able to avoid making some value queries,some items that are provably not part of the optimal allocation may never be discovered.This represents a big win in terms of minimizing the number of supply bun-dles that must be considered.The primary goal of this work is preference elicitation: reducing the number of value queries needed to solve,or approximately solve,the clearing problem.The motivation is that,in many applications,appropriate bids are compu-tationally difficult to generate.Our elicitation approach is covered in the next section.Throughout the paper we assume that the bidders bid truthfully rather than strategically.This is a reasonable as-sumption,for example,in multi-robot systems,where the robots are owned by the same real-world party.Further-more,it is well known that good strategy-proof exchanges cannot be constructed even in simple non-combinatorial ex-changes.Even with only one seller,only one buyer and only one item to be traded,it is impossible to design a mecha-nism that(1)motivates the parties to participate in the sensethat they can expect a nonnegative benefit from participat-ing,(2)allocates the item to the party that values it the most, and(3)does not require an external benefactor(Myerson&Satterthwaite1983).ApproachThe standard approach to the clearing problem(without preference elicitation)effectively makes all possible valuequeries up front,and then clears the exchange using com-plete information.In order to do better,we need to answer the following questions without complete information(i.e.,without knowing all of the bids):Bounding preferences:How can we use the preference constraints to infer the tightest possible bounds on bids?Best allocation search:What is the best allocation ac-cording to some metric(e.g.,the allocation with the high-est upper bound on welfare)?Optimality detection:Do we have enough information toprove that a given allocation is welfare maximizing?Query selection:What is the best next value query to make in order to reduce the total number of value queries needed tofind a provably welfare maximizing allocation? Bounding PreferencesHere we are looking for the tightest bounds we can infer on the welfare of a given allocation.First,we consider waysto use the free disposal constraint to bound the demand and reward in a bid.Given a production bid for a given supply bundle,the free disposal constraint dictatesthat any superset must have a corresponding demand.That is,is a lower bound on. If,then is an upper bound on.The free disposal constraint on reward works similarly.Thus, free disposal induces a lattice structure on supply bundles in subset/superset relationships,such that when the auctioneer gets the answer to a value query about,it can propagate bounds on demand and reward to subsets and supersets of .If a combination group contains items,we need to con-sider all possible sets of these items as supply bundles of production bids that we might make value queries for.The auctioneer keeps one node in memory for each supply bun-dle,representing the tightest known bounds on the demand and reward of the bid with that supply bundle.We refer to the data structure of nodes as the bid lattice2.Each node contains the following information about a supply bundle: 1.Demand interval:sets min and max,such that the demand bundle for bidder is guaranteed to satisfy min max.max may take on the special value,meaning that we have no upper bound information about demand.2.Reward interval:min and max .The reward is guar-anteed to satisfy min max .If minor max ,we have no information about the lower or upper bound,respectively.Because we discover items in the process of eliciting bids,each lattice starts empty,and new items are added as elicita-tion progresses.Thus we must support two basic operations on the bid lattice data structure:1.Adding a new item .For every node with supply bundlealready in the lattice,we add a new node with supply bundle and propagate bounds information from subsets of ,as explained below.2.Propagating bounds from a value query.When a query ismade for a supply bundle ,returningand ,we do the following:(a)Store the now exact bounds on :minmaxminmax(b)Propagate the bounds to supersets and subsets accord-ing to the constraints of free disposal:For every superset of ,set min minand max max.Similarly,for every subset of ,set maxmax and min min.Figure 1shows progressive states of a lattice correspond-ing to a combination group with two items:and .Rectangles in the figure are nodes,and bounds onand are shown as intervals .In stage (1),we see the lattice before any value queries are made.Stages (2a)and (2b)show bound propagation after the auctioneer queries about receives the bid.(update D max(1) No value queries made yet(2a) Store bounds (2b) Propagate boundsFigure 1:Propagating bounds from a value query.Best Allocation SearchIn an allocation,exactly one bid is accepted from each bid lattice.We use to denote the set of all bid lattices,and to denote the supply bundle of the bid that al-location accepts from lattice .Further,noting that each lattice belongs to a single bidder,we will use the notation and to mean and,where is the bidder for lattice .An allocation is feasible if demand does not exceed supply;that is,if.We define the welfare asif is feasible otherwiseWe also define the best case welfare best and worst case welfare worst of an allocation.To calculate these measures,we apply a selection function that fills in the miss-ing information in a bid to form a complete virtual bid,ac-cording to rules that we define:1.The best case selection function sets the virtual bid from bidder for to be min max .2.The worst case selection function sets the virtual bid from bidder for to be max min .Once a selection function has generated complete virtual bids,we calculate the welfare worst or best accord-ing to the definition already given.Note that the auctioneer can calculate these measures based on the information it has,and they satisfy worst best.Each selection function transforms the partial informa-tion in the bid lattice to complete information,represent-ing a best case or worst case scenario.Finding the al-location with the highest best case or worst case welfare is then an instance of the combinatorial exchange winner determination problem with complete information.This is an NP-hard and inapproximable problem.There are good existing algorithms for the related problem of clear-ing the combinatorial auction (Sandholm et al.2001a;Fujishima,Leyton-Brown,&Shoham 1999).The combi-natorial exchange is substantially harder,with the best algo-rithms clearing an exchange of a few hundred bids in a few seconds (Sandholm et al.2001b).Our current approach searches depth-first,branching on which bid is accepted from each bid lattice,with branch-and-bound pruning.This is structurally similar to some of the advanced algorithms;however,we did not invest the ef-fort to duplicate their sophisticated heuristics,because our experiments do not include timing results.Optimality DetectionOptimality detection is the problem of determining when wehave enough information to prove that an allocation is optimal.To do so,we need to show thatfor every allocation .Assuming and are feasible,this is equivalent to:We call the term for each in the above sum the advantage of over in lattice .What upper bound can we calculate on the advantage in order to prove ?One obvious answer is thatmaxminSumming over all lattices,this upper bound gives us the suf-ficient condition for superiority of worst best ,which is not surprising.But we can do better:if,free disposal dictates that ,so that we get the tightest possible boundmaxminA simple way to check that is optimal is to use this bound to compare to every other allocation.Unfor-tunately,there are exponentially many allocations to com-pare against.We can save some of this effort by defin-ing a new selection function called the upper bound advan-tage over(adv()),as follows:set the virtual bid for a supply bundle in lattice belonging to bidder to bemin adv,whereadvmax minmax minotherwiseThis selection has the property thatis provable given our current information if and only if adv.We can use our existing tech-niques to quicklyfind the allocation which maximizes adv.If adv,is optimal.This op-timality test is not only sound but also complete,meaning that itflags optimality of an allocation as soon as we haveenough information to prove it.We should also mention a much simpler optimality con-dition:find the allocation with highest best case wel-fare.If the auctioneer has tight bounds on the welfare of (worst best),then is optimal.This condi-tion is sound,but not complete.However,for our domain, it essentially always becomes true at the same time as the complete condition explained above,and is much faster to compute.Query SelectionQuery selection is the problem offinding the best next valuequery to make in order to reduce the total number of valuequeries needed tofind a provably welfare maximizing allo-cation.At this stage in our research,we are focusing on heuristic rules for query selection,and we do not yet have a general theory for how well the auctioneer can do. Currently,we refine the allocation with the highest best case welfare best.Refining means making a value query for one of the bids accepted by that we do not have complete information about.The intuition is that the auc-tioneer should continue working on until is either prov-ably optimal or no longer the best candidate.Among the bids accepted by,we currently query about the one clos-est to the consumer end of the supply chain.Overall Elicitation AlgorithmWe now have the building blocks for a full elicitation algo-rithm:1.Find the allocation that maximizes best.2.If is optimal,we are done.Check using either of thetwo optimality conditions described above.3.If desired,return an anytime result as follows:(a)The anytime answer is the allocation that maximizesworst.(b)Let be the allocation that maximizes adv.The solution quality of,defined as/OPT,hasa lower bound of worstinstance,the elicitation ratio line for 12items is generally above that for 10items).0.20.40.60.81050100150200250300e l i c i t a t i o n r a t i onumber of nodes in exchange5 items 10 items 12 itemsFigure 2:Scaling experiment:elicitation vs.exchange size.For the domain studied,elicitation allows significant and increasing savings in bidder effort as the exchange peting factors influence the ratio:1.When there are a larger number of nodes in each bid lattice,each query has the ability to affect more nodes through bounds propagation;this leads to a lower elicita-tion ratio as the number of bids increases.2.On the other hand,increasing numbers of items and nodes mean that there are more similar allocations whose cost is difficult to differentiate,so that more queries are needed to prove any one allocation optimal.Of these factors,(1)appears to be more important in our do-main when scaling the number of nodes.An apparent excep-tion is seen near the right end of the 5items line,where the elicitation ratio rises as the problems approach node satura-tion (i.e.,approach the maximum possible number of nodes given the number of items and the other fixed parameters of the domain).More research is needed to determine how domain parameters and query selection influence scaling.Anytime ExperimentThis experiment shows how the solution quality of the best allocation so far varies as a function of the number of value queries.After each query,the auctioneer reports the alloca-tion with the highest worst case welfare.Two measures of were taken:the actual quality /OPT,and the auc-tioneer’s provable lower bound on quality given its informa-tion so far.Median results over five runs are shown in Figure 3.Tests over a wide range of exchange sizes gave quali-tatively similar results.An alternative anytime scheme of reporting the allocation with highest best case welfare per-formed much worse.Most of the improvement in the lower bound on solution quality comes very near the end of the run,so approximation schemes that stop early based on the lower bound reaching some high quality will save only marginal effort.On the other hand,the actual quality typically reaches 1(optimality)about 20-50%more quickly than the lower bound.Thus,if we are willing to give up a provable quality0.20.40.60.811.200.10.20.30.40.50.60.70.80.91s o l u t i o n q u a l i t yproportion of nodes evaluated so far5 items, 26 nodesactual quality lower bound0.20.40.60.811.200.10.20.30.40.50.60.70.80.91s o l u t i o n q u a l i t yproportion of nodes evaluated so far12 items, 156 nodesactual quality lower boundFigure 3:Anytime experiment:solution quality vs.queries.bound,the auctioneer could do better.On repeated instances of similar problems,it could save significant effort by learn-ing how many queries are needed before it has a probably approximately optimal solution.Conclusions and Future ResearchWe have presented a design for a auctioneer agent that can construct and clear a combinatorial exchange using prefer-ence elicitation.Our design extends existing analyses of elicitation in the combinatorial auction to the combinatorial exchange.We also introduced the concept of item discovery during the process of preference elicitation.Our experimen-tal results,in a multi-robot exploration domain,show that elicitation allows significant and increasing savings in bid-der effort as the exchange scales.Tests of an anytime version of the elicitation algorithm show that we could save further effort by stopping early,if we are willing to accept a proba-bly approximately optimal solution.In future work,we hope to relax the single method of pro-duction constraint,allowing robots to have multiple plans for supplying the same items,with different corresponding demands and costs.We are also working on an architecture for coordinating multiple combinatorial exchanges.Each bidder communicates with one local exchange,and the lo-cal exchanges are themselves bidders in a larger exchange.In a distributed system,this architecture can minimize bot-tlenecks and delays due to communication latency.ReferencesBabaioff,M.,and Nisan,N.2001.Concurrent auctions across the supply chain.In ACM-EC .Conen,W.,and Sandholm,T.2001.Minimal preference elicitation in combinatorial auctions.In Proc.of the IJCAI Workshop on Economic Agents,Models,and Mechanisms .Dias,M.B.,and Stentz,A.2000.A free market architec-ture for distributed control of a multirobot system.In Proc.of IAS .Fujishima,Y .;Leyton-Brown,K.;and Shoham,Y .1999.Taming the computational complexity of combinatorial auctions:Optimal and approximate approaches.In Proc.of IJCAI ,548–553.Myerson,R.,and Satterthwaite,M.1983.Efficient mech-anisms for bilateral trading.Journal of Economic Theory 29.Nisan,N.,and Segal,I.2001.The communication com-plexity of efficient allocation problems.In Proc.of theDIMACS Workshop on on Computational Issues in Game Theory and Mechanism Design.Nisan,N.2000.Bidding and allocation in combinatorial auctions.In Proc.of ACM-EC.Parkes,D.,and Ungar,L.2000.Iterative combinatorial auctions:Theory and practice.In Proc.of AAAI,74–81. Parkes,D.1999.Optimal auction design for agents with hard valuation problems.In Proc.of the IJCAI Workshop on Agent-Mediated Electronic Commerce.Sandholm,T.;Suri,S.;Gilpin,A.;and Levine,D.2001a. CABOB:A fast optimal algorithm for combinatorial auc-tions.In Proc.of IJCAI,1102–1108.Sandholm,T.;Suri,S.;Gilpin,A.;and Levine,D.2001b. Winner determination in combinatorial auction generaliza-tions.In Proc.of AGENTS Workshop on Agent-Based Ap-proaches to B2B,35–41.Sandholm,T.2000.eMediator:A next generation elec-tronic commerce server.In Proc.of AGENTS,73–96. Sandholm,T.2002.Algorithm for optimal winner deter-mination in combinatorial auctions.Artificial Intelligence 135:1–54.Walsh,W.E.,and Wellman,M.P.1999.Efficiency and equilibrium in task allocation economies with hierarchical dependencies.In Proc.of IJCAI,520–526.Walsh,B.;Wellman,M.;and Ygge,binato-rial auctions for supply chain formation.In Proc.of ACM-EC.Wurman,P.R.,and Wellman,M.P.2000.AkBA:A pro-gressive,anonymous-price combinatorial auction.In Proc. of ACM-EC,21–29.。