Projects

These are the various projects that I and others have dreamed up. Please feel free to suggest your own to me, or modify the suggested projects. The projects are roughly honours level, but could also be seen as the first part of a Ph.D. Some of these projects may be co-supervised by other members of the department. If you're interested in any of the projects, email me: doug. aberdeen at anu. edu. au

Paid projects for undergraduates

These are programming orientated projects for good undergraduate students from mathematics, physics, engineering, or computer science. You will be paid an hourly rate to work on programming jobs, directed by an academic supervisor. This is an excellent opportunity for those interested in a research career, or simply someone that wants to apply their programming expertise to state of the art systems. Good workers can expect to be given paid opportunities over the a full year, moving from one project to the next.

Independent Projects For Honours/Ph.D/Summer Scholars

[SML_13] Traffic Light Singal Control with Reinforcement Learning

Applications such as controlling traffic lights, water flow, cell phone frequency allocation, and web services can be modelled as a network of co-operating devices. Finding the optimal control policy for each device in a distributed network is challenging. Reinforcement learning can do this naturally by using a single reward measure that represents the performance of the overall network. The independent devices learn to co-operate by jointly maximising this reward measure.

But this particular project focuses on traffic light optimisation. We've all been frustrated by traffic light timings that are clearly sub-optimal. But the truth is that human operators cannot deal with the complex interactions between all the intersections in a city size system. NICTA, in conjunction with the NSW Road Traffic Authority, have a project to try and improve the software that runs Sydney's (and many other cities) traffic light control system. We've made good progress with RL algorithms on a simple simulator. It's time to go to the next step and implement our algorithms on a realistic RTA approved simulation system. This will require implementing exactly the same protocol used on Sydney's roads (since the simulator assumes that). If the student is successful in this first phase they can progress to improving the algorithm with good control knowledge about traffic systems.

The project will involve a subsatial amount of coding in C++. It will also require learning (during the project) some of the mathematics behind policy-gradient reinformcement learning. The project will be based in Canberra, but working closely with project partners at the University of New South Wales The project will be based in Canberra, but working closely with project partners at the University of New South Wales. Since there is a industry partner, Students will definetely be required to sign over their IP for this project.

[SML_10] Ro-Sham-Bo player

Ro-Sham-Bo, or Rock, Paper, Scissors, is a classic game often used to make decisions because it is completely luck based.... Or is it? If you assume your opponent has a strategy it is suddenly better to have a strategy yourself.

This project will extend our early work on the Penny Matching and Ro-Sham-Bo (play online). An initial design would be a simple extension of the Mind-reader program.

This project will suit undergrad/honours level student with basic programming experience in any language, although students with C++ experience will able to take advantage of existing code. It's a chance to work with state of the art algorithms for modelling dynamical systems, called "Predicitive State Representations".

The student will work independently, with weekly supervisor meetings.

Projects with he co-operation of the Defence Science Technology Organisation

[SML_08] Probabilistic Temporal Planning (with Olivier Buffet and Sylvie Thiebaux)

Planning is the process of finding which sequence of decisions to take so as to achieve a given goal. We are currently working on a large planning domain aiming at scheduling tasks with probabilistic outcomes. To our knowledged, the tool which has been developped is the first one handling time, resources and probabilities at the same time.

Various questions can be addressed in this framework. Improving the planning algorithms is a major issue, may that be in the framework of Markov Decision Processes or more classical planning. Other issues as plan visualization or human-machine interaction are of interest, as the plan should be easy to understand and the user may need to give feedback to the software (by setting new constraints for planning).

There are several sub-projects available. We highly encourage interested students to come and talk to us to discuss specific possibilities. Software used in this project may end up being use by the Defence Science Technology Organisation.

The student will work independently, with weekly supervisor meetings. You may be required to sign over any intellectual property you generate to NICTA. Sub-projects:

Faster heuristic methods for search initialisation
Reducing action spaces
Planning graph approaches
Multi-agent planning

[1] D. Aberdeen and S. Thiebaux and L. Zhang, "Decision-Theoretic Military Operations Planning", in Proceedings of the Fourteenth International Conference on Automated Planning and Scheduling (ICAPS'04), Whistler, Canada, June 2004.

The views and opinions expressed on this web page are not necessarily those of NICTA or the Australian National University. Any HTML or image from this page may be copied and re-used freely but must not be sold.

Feedback:Doug.Aberdeen AT anu.edu.au