Handout #5 - Project 3

• A lot of resources are available on-line and in the library. It is great to use these, but please always cite them in your projects.

Objective

In this project you will add dynamic activity to your site in the form of stock trading. The trading rules below are meant to be only a very vague approximation of what happens in a real stock market. The purpose being to implement something with interesting behavior, not to simulate reality.

Database tables & fields

You will have to augment the database tables from project 2 in order to implement the stock trading. We strongly suggest you add the following tables and fields (more may be needed). Before deviating from this structure, think carefully whether you really need to.

Accounts table:

• user id
• user password
• user full name
• administrator flag (true/false)
• cash balance

Orders table:

• order id
• ticker
• user id
• type (buy, sell)
• quantity
• limit price (null, if none)
• date placed
• status (pending, expired, fulfilled, split)
• date retired (null, if pending)
• transaction price (null, if not fulfilled)
• parent order id (null, if none)

Holdings table:

• ticker
• user id
• quantity

Web pages

Your site will feature the following pages for its users (in addition to the pages from project 2):

• Home page: same as in project 2, but augmented with additional login fields: user name, password, and "login" button. Once a user is logged-in, you must track the user through all web pages.
• Account page: shows the current account state
  • User id and full name
  • Cash balance
  • Links to transaction history and order status pages
  • Holdings showing for each stock: ticker, company name, quantity, price (mid of buy&sell), value (qty*price)
  • Order submission form with fields to enter: ticker, order type (buy/sell), quantity, optional limit price, "submit" and "clear" buttons
  • Quote display & request form: ticker field and "get quote" button, when submitted, redisplays the account page, but adds the current buy & sell prices for the requested company (e.g. just above the quote request fields)
  • This page is only accessible to logged-in users
• Transaction history pages: show retired orders and outcomes for current account
  • Each page shows at most 20 orders, and a "page 1 2 3 4 5 ..." index provides links to all pages of the history
  • Initial page shown has most recent transactions
  • For each order, show: order id, ticker, company name, transaction type, quantity, limit price, date&time posted, status, date&time retired, transaction price
  • If orders have been split, group the "splittings" with the parent order, i.e. show something one can understand
  • This page is only accessible to logged-in users
• Order status pages: show pending orders for current account
  • For each order, show: order id, ticker, company name, transaction type, quantity, limit price, date&time posted
  • Each page shows at most 20 orders, and a "page 1 2 3 4 5 ..." index provides links to all pages of the status
  • Initial page shown has most recent orders
  • If orders have been split, group the "splittings" with the parent order, i.e. show something one can understand
  • This page is only accessible to logged-in users
• Company page: same page as for project 2, but augmented:
  • show current buy/sell quotes, see "how to compute quotes" below
  • link to order book page
• Chart pages: same pages as for project 2, but ensure one can select a "1 day" time period
• Order book page: displays the order book for a company
  • display two tables side-by-side showing buy and sell orders, respectively
  • for each order, show limit price and quantity
  • the buy table is sorted with the highest price at the top, the sell table with the lowest
• Admin page: an user flagged as "administrator" has access to this page and can manage other accounts (primarily to initialize new accounts)
  • field to enter the user id of the account to display
  • user id, password, full name, and cash balance of selected account
  • field to enter a new balance, "submit" button
  • button to clear current holdings, transaction history, and pending orders
  • fields to create a new account with an initial balance
  • This page is only accessible to logged-in users

In addition, your site must have the following pages for robots that submit orders and verify the operation of the stock market. For simplicity, these pages are not password protected. Each page must be of content-type text/plain (not HTML!), and consists of a table, one row per line, and each field separated by a comma. The fields must be in the order shown in the descriptions below. Date&time fields must be represented [NEW] either as YYYY-MM-DD HH:MM:SS.S or in milliseconds since "the epoch". For example:

tve, Thorsten von Eicken, false, 1000000.00
josep, Josep Blanquer, true, 0.01

• Robot account status:
  • URL: /robots/accounts
  • lists all accounts, sorted alphabetically by user id
  • fields: user id, user name, admin flag (values true or false), balance (in dollars and cents as shown in the example above)
• Robot quote:
  • URL: /robots/quotes or /robots/quotes?ticker
  • provide quote for requested company, or for all companies sorted alphabetically by ticker if no ticker is specified
  • fields: ticker, buy price, buy quantity, sell price, sell quantity (see "how to compute quotes" below)
  • note: the quantities are the number of shares offered at the corresponding price
• Robot order book: 
  • URL: /robots/orders?ticker [NEW] (note: NOT /robots/orders?ticker=ticker !!!)
  • lists all pending orders for the specified company, or all orders for all companies if no ticker is specified
  • sort alphabetically by ticker, and then by date&time placed
  • fields: order id, ticker, date&time placed, user id, order type (values buy or sell), quantity, limit price
  • [NEW] the limit price field must be present and can be "none", "null", or empty if there is no limit price
  • [NEW] example result lines:
    9602,BND,2001-06-03 23:53:33.0,acct09,buy,7.000,1.6500,fulfilled,2001-06-04 00:02:59.0,1.6000,none
    9608,BND,2001-06-04 00:02:51.0,acct03,sell,10.000,1.5500,split,2001-06-04 00:02:59.0,null,none
    9609,BND,2001-06-04 00:02:51.0,acct03,sell,7.000,1.5500,fulfilled,2001-06-04 00:02:59.0,1.6000,9608
• Robot transaction history:
  • URL: /robots/transactions?ticker [NEW] (note: NOT /robots/transactions?ticker=ticker !!!)
  • lists all retired transactions for the specified company
  • sort alphabetically by date&time placed
  • fields: order id, ticker, date&time placed, user id, order type (values buy or sell), quantity, limit price, status [NEW] (values s|split|f|fulfilled|e|expired), date retired, transaction price, parent order id
  • [NEW] the date retired/transaction price and parent order id fields must be present and can be "none", "null", or empty if they are not needed
  • [NEW] example result lines:
    9602,BND,2001-06-03 23:53:33.0,acct09,buy,7.000,1.6500,fulfilled,2001-06-04 00:02:59.0,1.6000,none
    9608,BND,2001-06-04 00:02:51.0,acct03,sell,10.000,1.5500,split,2001-06-04 00:02:59.0,null,none
    9609,BND,2001-06-04 00:02:51.0,acct03,sell,7.000,1.5500,fulfilled,2001-06-04 00:02:59.0,1.6000,9608
• Robot stock price table:
  • URL: /robots/chart?ticker
  • lists the closing prices for a stock
  • for all days on record show the closing price of the stock
  • fields: date (DD-MM-YYYY), price
• Robot order entry:
  • URL: /robots/entry?url-encoded-parameters
  • parameters: ticker, userid, type (buy/sell), quantity, optional limit price
  • example: /robots/entry?ticker=aapl&userid=tve&type=buy&quantity=100&price=123.45
  • the server should return a text/plain page stating: modified
    • "OK" followed by the assigned order id
    • "INSUFFICIENT FUNDS"
    • "ERROR" followed by a second line with an english explanation of the error
• Robot initialization: modified
  • URL: /robots/init
  • This request is a POST that creates a number of accounts and returns the database to its initial state
  • The database is returned to the state after loading the historical quotes from project 1. I.e. all orders, transactions, quotes,  etc. that occurred after March 30th 2001 must be removed
  • All accounts have all their holdings cleared
  • The post contains a list of accounts to be created (or initialized, if they already exist), for each account the initial cash balance, and holdings in one stock are specified
  • fields of the post: user id, user name, admin flag (values true or false), cash balance, ticker, quantity
  • the server should return a text/plain page stating (as appropriate) "OK" or "ERROR" followed by a newline. In case of ERROR, a second line should have an english explanation of the error.

Notes:

• You will need pages in addition to the above, especially for error handling
• All stock prices are in dollars and thousands (i.e. three decimal digits), while all account balances are in dollars and cents (two decimal digits). Round to the nearest after every transaction when adjusting the balance.

How to settle orders

Modified: Your order settlement is event driven and is triggered whenever a new order is entered into the system. First, each new order is assigned an "order id", which is a monotonically increasing integer, which captures the order which which orders (uhh) are entered. The new order is then matched against all orders of opposite type (buy vs. sell) that are in the company order book. The following matches can occur:

1. Both orders have a limit price, the buy price is lower than the sell price
   • No match, continue
2. Both orders have a limit price, the buy price is higher than the sell price
   • The orders "cross", a transaction occurs
   • The transaction price is the mid-point between the two limit prices
   • The quantity is the lower of the two quantities
   • If that leaves a residual quantity on one side, that order is split into two orders:
     • The outcome of the original order is "split"
     • Two new orders are created with the original order as "parent", with all the same field values as the parent, except that one's outcome is "fulfilled" for the transaction price and quantity, while the other remains open for the remaining quantity
     • Any remaining open orders may have to be matched in-turn
3. One order has a limit price, the other doesn't
   • The orders "cross", a transaction occurs
   • The transaction price is the limit price
   • Proceed as in case #2
4. Neither order has a limit price
   • The orders "cross", a transaction occurs
   • The transaction price is the same as the price of the previous transaction for that company
   • Proceed as in case #2

Any order that "falls through" the matching process (i.e. cannot be matched) is added to the order book.

At the end of a day (at midnight: we assume 24/7 trading), all order books are cleared: all pending orders receive an outcome of "expired".

Example of an order settlement with a split:

1. Suppose the order book initially contains one order:
   • id=1, ticker=AAPL, type=sell, qty=100, limit_price=$50, status=pending
2. Now a buy order is placed:
   • id=2, ticker=AAPL, type=buy, qty=500, limit_price=$54, status=pending
3. Settlement applies rule #2
   • The transaction price is $52 and the quantity is 100
4. The original order is now fulfilled:
   • id=1, ticker=AAPL, type=sell, qty=100, limit_price=$50, status=fulfilled, transaction_price=$52
5. The second order is only partially fulfilled and has to be split. This results in the following three orders:
   • id=2, ticker=AAPL, type=buy, qty=500, limit_price=$54, status=split
   • id=3, ticker=AAPL, type=buy, qty=100, limit_price=$54, status=fulfilled, transaction_price=$52, parent=2
   • id=4, ticker=AAPL, type=buy, qty=400, limit_price=$54, status=pending, parent=2

Note on ordering constraints: modified

Orders have to be processed strictly according to the assigned order id. So this must be a sequential process. However, there is some room for adjustment in assigning the order id. A client can only observe three events: the time at which the HTTP request is sent (Treq), the time at which the HTTP response is received (Tresp), and the order id that was assigned (Oid). The only invariants that the server must maintain are:

Tresp1 < Treq2  =>  Oid1 < Oid2
Oid1 < Oid2  =>  Treq1 < Tresp2

How to compute quotes

The quotes displayed on the company page and on the robot quote page are detailed quotes that show the top of the order book, and not just  a single number representing the price of the last transaction. This full quote consists of 5 items:

1. the ticker symbol
2. the highest limit price of all buy orders in the order book
3. the quantity to be bought at that price
4. the lowest limit price of all sell orders in the order book
5. the quantity to be sold at that price 

For example:

AAPL: buy 100 @ $100.000 : sell 1000 @ $110.550

indicates that the best buy offer is at $100 and that 100 shares are being asked for at that price, while the best sell offer is at $110.550 and that 1000 shares are available at that price.

In more detail, the prices and quantities are computed as follows (apply these steps for buy and sell orders separately):

• if the order book has at least one order with a limit price, then
  • compute p = the "best" price p
  • compute q = the sum of the quantities of all orders at price p
  • if the order book has at least one order without limit price, then
    • display the price p
    • display the sum of q and the quantities of all orders without limit price
  • else
    • display the price p and the quantity q
• Else (no order with a limit price exists)
  • if the order book has at least one order without limit price, then
    • display the price of the last transaction
    • display the sum of the quantities of all orders without limit price
  • else
    • display the price of the last transaction and the quantity 0

For the historical quote information (i.e. filling in the tables inherited from project 1), the closing price is the price of the last transaction of the day. The opening price is the same as the previous day's closing price. The high and low prices are the max/min transaction prices of the day.

How to compute time

The order settling algorithm clearly must take the time at which orders are placed into account. In addition, some bookkeeping needs to be done at the end of a "day". In order to avoid having to run benchmarks for days, we will use a fake clock that runs 1000 times faster than real-time. So simulating one day takes 86.4 seconds. In unix, the date&time are kept as milliseconds since "epoch" (sometime in 1970), call current_date the current number of real seconds since the epoch. In addition, call start_date the value of current_date at the time the last robot initialization page request was received. Then the fake_date is computed as follows:

fake_date = start_date + ( current_date - start_date ) * 1000

I.e., we make time run 1000 times faster since the last robot init page request. The server must use this fake time throughout its operation.

What to turn in modified

Write a "one page" project overview describing the overall structure of your code. Describe the presentation layer (using WebMacro?), the business logic, and the storage/DB layer. Any special features you are using? Special algorithms, tricks, etc? Any packages you found on the web that are useful?

What's coming next

Project 4 will have a very simple problem statement: run your web service on 3 machines at the same time, all using the same database on bugatti (4th machine). We will load-balance incoming requests round-robin to your 3 servers. Provide the best performance you can, and make your service resilient to server and machine failures (i.e. we will kill your processes and/or halt the machines).