It reads top to bottom
Names first, then the steps in the order you want them, then the answer.
customers {
paid = .orders.where { .status == "paid" }
where { paid.count() > 0 }
select { name }
}Owl is a database for objects that point at each other.
You follow the references. You never write a join.
H to hoot
The language · OQL
Seven short chapters, from describing your data to questions that are hard in SQL. Every one runs on the same six orders, and every answer shown came from a real Owl server.
orders
.where { .status == "paid" }
.sort { .total.desc }
.limit(3)
.select { id, total }where keeps the rows that pass a test.sort puts the largest total first.limit keeps the first three.select chooses what each row of the answer holds.{ id: "o1", total: 125.0 }
{ id: "o6", total: 97.0 }
{ id: "o2", total: 85.0 }orders
.where { .customer.country == "NG" }
.sort { .id.asc }
.select { id, who: .customer.name, total }.customer follows it. There is no join to write and no key to match.{ id: "o1", who: "Ada", total: 125.0 }
{ id: "o2", who: "Bola", total: 85.0 }
{ id: "o3", who: "Ada", total: 24.0 }
{ id: "o6", who: "Ada", total: 97.0 }customers.sort { .id.asc }.select {
name,
orders: .orders.count(),
spent: .orders
.where { .status == "paid" }
.sum(.total)
}.orders is every order that points here. It behaves like any collection.{ name: "Ada", orders: 3, spent: 222.0 }
{ name: "Bola", orders: 1, spent: 85.0 }
{ name: "Chen", orders: 2, spent: 52.0 }orders
.group_by { .status }
.select {
status, n: count(), revenue: sum(.total)
}
.sort { .revenue.desc }group_by makes one pile for each status.select, count() and sum() measure a whole pile.{ status: "paid", n: 4, revenue: 359.0 }
{ status: "cancelled", n: 1, revenue: 40.0 }
{ status: "open", n: 1, revenue: 24.0 }categories
.where { .id == "home" }
.walk(.children)
.products
.sort { .id.asc }
.select { name, price }walk follows .children one level down.{ name: "Kettle", price: 40.0 }
{ name: "Chef knife", price: 85.0 }
{ name: "Trowel", price: 12.0 }orders
.sort { .customer.id.asc, .total.desc }
.partition_by { .customer }
.select {
id, who: .customer.name, total,
place: row_number()
}
.where { .place == 1 }partition_by lets each customer's orders be counted separately.row_number() is a row's position within its own customer.{ id: "o1", who: "Ada", total: 125.0, place: 1 }
{ id: "o2", who: "Bola", total: 85.0, place: 1 }
{ id: "o4", who: "Chen", total: 52.0, place: 1 }schema customers {
id: String,
name: String,
country: String
}
schema orders {
id: String,
status: String,
total: Float,
customer: Ref(customers),
items: List({
product: Ref(products),
qty: Integer
})
}A schema names a collection and the type of each field.
Two forms
Every query can be written as a chain of steps or as a block. They are the same language and give the same answer. Pick the one that reads better.
orders
.where { .status == "paid" }
.sort { .total.desc }
.limit(3)
.select { id, who: .customer.name, total }orders {
where { .status == "paid" }
sort { .total.desc }
limit 3
select { id, who: .customer.name, total }
}Both start from a collection. A chain hangs steps off it with dots. A block opens a pair of braces and puts the steps inside.
{ id: "o1", who: "Ada", total: 125.0 }
{ id: "o6", who: "Ada", total: 97.0 }
{ id: "o2", who: "Bola", total: 85.0 }A block can name something once and use it wherever it is needed. That is the one thing it has that a chain does not.
customers
.sort { .id.asc }
.select {
name,
orders: .orders.where { .status == "paid" }.count(),
spent: .orders.where { .status == "paid" }.sum(.total)
}customers {
paid = .orders.where { .status == "paid" }
sort { .id.asc }
select {
name,
orders: paid.count(),
spent: paid.sum(.total)
}
}{ name: "Ada", orders: 2, spent: 222.0 }
{ name: "Bola", orders: 1, spent: 85.0 }
{ name: "Chen", orders: 1, spent: 52.0 }customers
.where {
.orders.where { .status == "paid" }.sum(.total) > 80
}
.sort { .id.asc }
.select {
name,
orders: .orders.where { .status == "paid" }.count(),
spent: .orders.where { .status == "paid" }.sum(.total)
}customers {
paid = .orders.where { .status == "paid" }
spent = paid.sum(.total)
where { spent > 80 }
sort { .id.asc }
select {
name,
orders: paid.count(),
spent: spent
}
}{ name: "Ada", orders: 2, spent: 222.0 }
{ name: "Bola", orders: 1, spent: 85.0 }Each customer's paid orders, counted. So far the chain is no longer than the block.
customers
.sort { .id.asc }
.select {
name,
orders: .orders.where { .status == "paid" }.count()
}customers {
paid = .orders.where { .status == "paid" }
sort { .id.asc }
select { name, orders: paid.count() }
}{ name: "Ada", orders: 2 }
{ name: "Bola", orders: 1 }
{ name: "Chen", orders: 1 }Names first, then the steps in the order you want them, then the answer.
customers {
paid = .orders.where { .status == "paid" }
where { paid.count() > 0 }
select { name }
}A block finishes with select, or with a closing step such as count(). Nothing comes after it.
customers {
select { name, country }
where { .country == "NG" }
}Owl says select must be the final entry in a scoped query block
A name cannot be given twice, and it cannot reuse the name of the collection the block is about.
customers {
paid = .orders.where { .status == "paid" }
paid = .orders
select { name }
}Owl says Scoped binding 'paid' is already defined
A block is a whole query. Its names do not exist outside it, and nothing can be chained on after it closes.
customers {
paid = .orders.where { .status == "paid" }
select { name }
}.where { paid.count() > 0 }Owl says Expected end of query, found '.'
Inside a nested query, . is the nested row. The outer collection's name still reaches the row the block is on.
customers {
sort { .id.asc }
select {
name,
n: orders
.where { .customer == customers.id }
.count()
}
}For a short run of steps, and for anything that carries on after select, such as sorting by a field the select just worked out.
When the query uses the same fact more than once, or when naming the parts makes a long query easier to read.
Functions
A function gives a name to a measure, a rule, a test or a path. You write it in OQL, Owl keeps it in the database, and from then on every query can use the word.
fn tier(spent) {
if spent >= 200 then "gold"
else if spent >= 80 then "silver"
else "bronze"
}customers.sort { .id.asc }.select {
name,
spent: spent(*),
tier: tier(spent(*))
}tier takes what spent worked out, so the rule for a gold customer is written in one place.{ name: "Ada", spent: 222.0, tier: "gold" }
{ name: "Bola", spent: 85.0, tier: "silver" }
{ name: "Chen", spent: 52.0, tier: "bronze" }fn is_big(o: orders) -> Bool {
o.status == "paid" && o.total >= 90
}orders
.where { is_big(*) }
.sort { .id.asc }
.select { id, total }where keeps the rows it says yes to, and the query reads like the question you asked.{ id: "o1", total: 125.0 }
{ id: "o6", total: 97.0 }fn spread(values) {
values.max() - values.min()
}customers.sort { .id.asc }.select {
name,
spread: .orders.total.spread()
}.orders.total.spread(), spread(.orders.total), or on the end of a query.{ name: "Ada", spread: 101.0 }
{ name: "Bola", spread: 0.0 }
{ name: "Chen", spread: 12.0 }fn also_ordered(o: orders) {
orders.where {
.customer == o.customer && .id != o.id
}
}orders
.where { .id == "o1" }
.also_ordered()
.sort { .id.asc }
.select { id, total }{ id: "o3", total: 24.0 }
{ id: "o6", total: 97.0 }fn spent(c: customers) {
c.orders
.where { .status == "paid" }
.sum(.total)
}customers.sort { .id.asc }.select {
name,
spent: spent(*)
}{ name: "Ada", spent: 222.0 }
{ name: "Bola", spent: 85.0 }
{ name: "Chen", spent: 52.0 }* hands the function the whole current row. Inside, c.orders walks the reference back, the same as anywhere else in OQL.
A function lives in the database, not in each program that talks to it. Change the rule once and every caller gets the new answer.
fn tier(spent) {
if spent >= 200 then "gold"
else if spent >= 80 then "silver"
else "bronze"
}Ada gold
Bola silver
Chen bronze
{ "who": "Ada",
"tier": "gold" }
{ "who": "Bola",
"tier": "silver" }
{ "who": "Chen",
"tier": "bronze" }gold1
silver1
bronze1
Move the slider. This page works the rule out itself to show the effect. On a real server, defining tier again with 250 turned Ada from gold to silver in every query that used it.
Kept with your data. A definition is written to the database like any other row.
Still there tomorrow. It survives a restart.
Replaced by saying it again. The latest definition is the one Owl uses.
Undone with its transaction. Roll the transaction back and the old definition returns.
The last row is written in OQL, the same way yours are. Define one of the same name and yours takes its place.
trimupperlowerlengthsubstringconcatreplacesplitstarts_withends_withcontainsmatchespad_leftpad_right
absroundfloorceilsqrtpowminmaxloglnexprandom
nowtodaydays_agohours_agoformat_dateparse_dateyearmonthdayweekdayadd_daysdiff_daysdays_untildays_since
coalesceif_nullto_stringto_intto_floatto_booltype_of
variancestddevvariance_popstddev_popmode
Writing
Writes start from a collection, like reads do, and use the same where. The schema holds the rules, and Owl refuses any write that would break one.
products
.where { .id == "knife" }
.update { stock -= 1 }stock -= 1 is short for stock: .stock - 1. A path such as address.city: "Ibadan" changes one part of a nested object and leaves the rest.{ id: "knife", name: "Chef knife", price: 85.0, stock: 4 }customers.upsert on id {
id: "chen",
name: "Chen W."
}on id names the field that decides. No row with that id: Owl inserts. One row: Owl updates it and keeps every field you left out.{ id: "chen", name: "Chen W.", email: "chen@shop.test",
address: { city: "Singapore" } }orders
.where { .status == "cancelled" }
.delete(){ id: "o2", customer: "bola", product: "knife",
qty: 1, status: "cancelled" }customers.insert {
id: "chen",
name: "Chen",
email: "chen@shop.test",
address: { city: "Singapore" }
}customers
{ id: "chen", name: "Chen", email: "chen@shop.test",
address: { city: "Singapore" } }Give insert a list in square brackets and it adds them all, or none: one bad row in the list and nothing is added.
You say once what must always be true. Owl checks it on every write, whichever program is asking.
schema customers {
id: String,
name: String,
email: String unique
}
schema products {
id: String,
price: Float,
stock: Integer,
check(.price >= 0 && .stock >= 0)
}
schema orders {
id: String,
customer: Ref(customers),
qty: Integer
}
schema payments {
id: String,
order: Ref(orders) on delete cascade
}Try to break it
products
.where { .id == "kettle" }
.update { price: -5.0 }orders.insert {
id: "o9", customer: "nobody",
product: "kettle", qty: 1,
status: "open"
}orders.insert {
id: "o8", customer: "ada",
product: "kettle", qty: "two",
status: "open"
}customers
.where { .id == "ada" }
.delete()orders
.where { .id == "o1" }
.delete()customers.insert {
id: "eve", name: "Eve",
email: "ada@shop.test",
address: { city: "Accra" }
}Owl says 'customers' document 'eve' has email "ada@shop.test", which must be unique, and another document already has it
The test and the change happen together, so two buyers cannot both take the last kettle. Most updates need nothing more than this.
products
.where { .id == "kettle" && .stock > 0 }
.update { stock -= 1 }When changes only make sense together, put them in a transaction block. If any one fails, none of them happened.
transaction {
orders.insert {
id: "o4", customer: "ada",
product: "kettle", qty: 2, status: "paid"
}
products
.where { .id == "kettle" }
.update { stock -= 2 }
}orders
o1o2o3o4kettle stock
1the order wants 2Only one kettle is left, and Ada wants two. Run it both ways.
The engine
There is no other database underneath. Owl's own engine stores the data, and it knows your schema and your references, so it can use both.
Every object has a 64-bit identity that says nothing about where it is stored. Owl can move the data and every reference still holds.
Field names are stored once, not in every record. Ask for two fields and Owl decodes two fields.
Every commit is checksummed and synced to disk before any reader sees it. After a crash, Owl loads its last checkpoint and replays the rest.
Every reference gets an index for the way back. You can add ordered, unique, composite and vector indexes.
Evidence
Two suites give Owl and SQLite the same data, the same indexes and the same durability, then require the same rows back. Only then is anything timed. One suite is an ordinary shop. The other is built to be hard.
The shop suite has sixteen queries. Its check fails unless both engines return the same rows for every one.
Read the suitesRun it
# 1. Start a server (needs only a Rust toolchain)
cargo run --release -p owl-server -- --data shop.owl
# 2. Declare two collections and add a row to each
curl 127.0.0.1:7878/v1/script --json '{"source": "schema customers { id: String, name: String }\nschema orders { id: String, customer: Ref(customers), total: Float }\ncustomers.insert { id: \"c1\", name: \"Ada\" }\norders.insert { id: \"o1\", customer: \"c1\", total: 12.5 }"}'
# 3. Follow the reference from order to customer
curl 127.0.0.1:7878/v1/query --json '{"query": "orders.select { id, who: .customer.name }"}'
{"rows":[{"id":"o1","who":"Ada"}]}Join in
Owl is open source under the MIT license, and it is early. If you like where it is going, here are three ways to help, from ten seconds to a weekend.
A star costs nothing, and it is how other people come across a project this new.
Star Owl on GitHubRun the three commands above against data shaped like yours. A query Owl gets wrong, or cannot say, is the most useful report there is.
Open an issueThe priority right now is getting storage right. Good places to start:
crash testsformat fuzzingthe record codeckey encodingindex testsrepeatable benchmarks
How to contribute