@pnkfelix
), Mozillahttp://bit.ly/1LQM3PS
No segmentation faults
No undefined behavior
No data races
msg passing via channels
shared state via Arc
and atomics, Mutex
, etc
use native threads... or scoped threads... or work-stealing...
It's awesome!
(Were prior slides really not a sufficient answer?)
oh, maybe you meant ...
"Our mission is to ensure the Internet is a global public resource, open and accessible to all. An Internet that truly puts people first, where individuals can shape their own experience and are empowered, safe and independent."
1.0 release was back in May 2015
Rolling release cycle (up to Rust 1.7 as of March 2nd 2016)
Open source from the begining https://github.com/rust-lang/rust/
Open model for future change (RFC process) https://github.com/rust-lang/rfcs/
Awesome developer community (~1,000 people in #rust
, ~250 people in #rust-internals
, ~1,300 unique commiters to rust.git)
Sharing | Stuff |
---|---|
Sharing capabilities | (Language stuff) |
Sharing work | (Parallelism stuff) |
Sharing code | (Open source distribution stuff) |
fn sequential_web_fetch() {
use hyper::{self, Client};
use std::io::Read; // pulls in `chars` method
let sites = &["http://www.eff.org/", "http://rust-lang.org/",
"http://imgur.com", "http://mozilla.org"];
for &site in sites { // step through the array...
let client = Client::new();
let res = client.get(site).send().unwrap();
assert_eq!(res.status, hyper::Ok);
let char_count = res.chars().count();
println!("site: {} chars: {}", site, char_count);
}
}
(lets get rid of the Rust-specific pattern binding in for
; this is not a tutorial)
fn sequential_web_fetch() {
use hyper::{self, Client};
use std::io::Read; // pulls in `chars` method
let sites = &["http://www.eff.org/", "http://rust-lang.org/",
"http://imgur.com", "http://mozilla.org"];
for site_ref in sites { // step through the array...
let site = *site_ref; // (separated for expository purposes)
{ // (and a separate block, again for expository purposes)
let client = Client::new();
let res = client.get(site).send().unwrap();
assert_eq!(res.status, hyper::Ok);
let char_count = res.chars().count();
println!("site: {} chars: {}", site, char_count);
}
}
}
fn concurrent_web_fetch() -> Vec<::std::thread::JoinHandle<()>> {
use hyper::{self, Client};
use std::io::Read; // pulls in `chars` method
let sites = &["http://www.eff.org/", "http://rust-lang.org/",
"http://imgur.com", "http://mozilla.org"];
let mut handles = Vec::new();
for site_ref in sites {
let site = *site_ref;
let handle = ::std::thread::spawn(move || {
// block code put in closure: ~~~~~~~
let client = Client::new();
let res = client.get(site).send().unwrap();
assert_eq!(res.status, hyper::Ok);
let char_count = res.chars().count();
println!("site: {} chars: {}", site, char_count);
});
handles.push(handle);
}
return handles;
}
site: http://www.eff.org/ chars: 42425
site: http://rust-lang.org/ chars: 16748
site: http://imgur.com chars: 152384
site: http://mozilla.org chars: 63349
(on every run, when internet, and sites, available)
site: http://imgur.com chars: 152384
site: http://rust-lang.org/ chars: 16748
site: http://mozilla.org chars: 63349
site: http://www.eff.org/ chars: 42425
(on at least one run)
fn sequential_web_fetch_2() {
use hyper::{self, Client};
use std::io::Read; // pulls in `chars` method
let sites = &["http://www.eff.org/", "http://rust-lang.org/",
// ~~~~~ `sites`, an array (slice) of strings, is stack-local
"http://imgur.com", "http://mozilla.org"];
for site_ref in sites {
// ~~~~~~~~ `site_ref` is a *reference to* elem of array.
let client = Client::new();
let res = client.get(*site_ref).send().unwrap();
// moved deref here ~~~~~~~~~
assert_eq!(res.status, hyper::Ok);
let char_count = res.chars().count();
println!("site: {} chars: {}", site_ref, char_count);
}
}
fn concurrent_web_fetch_2() -> Vec<::std::thread::JoinHandle<()>> {
use hyper::{self, Client};
use std::io::Read; // pulls in `chars` method
let sites = &["http://www.eff.org/", "http://rust-lang.org/",
// ~~~~~ `sites`, an array (slice) of strings, is stack-local
"http://imgur.com", "http://mozilla.org"];
let mut handles = Vec::new();
for site_ref in sites {
// ~~~~~~~~ `site_ref` still a *reference* into an array
let handle = ::std::thread::spawn(move || {
let client = Client::new();
let res = client.get(*site_ref).send().unwrap();
// moved deref here ~~~~~~~~~
assert_eq!(res.status, hyper::Ok);
let char_count = res.chars().count();
println!("site: {} chars: {}", site_ref, char_count);
// Q: will `sites` array still be around when above runs?
});
handles.push(handle);
}
return handles;
}
Let's buy a car
let money: Money = bank.withdraw_cash();
let my_new_car: Car = dealership.buy_car(money);
let second_car = dealership.buy_car(money); // <-- cannot reuse
money transferred into dealership
, and car transferred to us.
Let's buy a car
let money: Money = bank.withdraw_cash();
let my_new_car: Car = dealership.buy_car(money);
// let second_car = dealership.buy_car(money); // <-- cannot reuse
money transferred into dealership
, and car transferred to us.
my_new_car.drive_to(home);
garage.park(my_new_car);
my_new_car.drive_to(...) // now doesn't work
(can't drive car without access to it, e.g. taking it out of the garage)
Let's buy a car
let money: Money = bank.withdraw_cash();
let my_new_car: Car = dealership.buy_car(money);
// let second_car = dealership.buy_car(money); // <-- cannot reuse
money transferred into dealership
, and car transferred to us.
my_new_car.drive_to(home);
garage.park(my_new_car);
// my_new_car.drive_to(...) // now doesn't work
(can't drive car without access to it, e.g. taking it out of the garage)
let my_car = garage.unpark();
my_car.drive_to(work);
...reflection time...
(copying data like integers, and characters, and .mp3's, is "free")
("On sense and reference" -- Gottlob Frege, 1892)
If ownership were all we had, car-purchase slide seems nonsensical
my_new_car.drive_to(home);
Does this transfer home
into the car?
Do I lose access to my home, just because I drive to it?
We must distinguish an object itself from ways to name that object
Above, home
cannot be (an owned) Home
home
must instead be some kind of reference to a Home
We can solve any problem by introducing an extra level of indirection
-- David J. Wheeler
Ownership enables: | which removes: |
---|---|
RAII-style destructors | a source of memory leaks (or fd leaks, etc) |
no dangling pointers | many resource management bugs |
no data races | many multithreading heisenbugs |
Do I need to take ownership here, accepting the associated resource management responsibility? Would temporary access suffice?
Good developers ask this already!
Rust forces function signatures to encode the answers
(and they are checked by the compiler)
Move | Copy | Copy if T:Copy |
---|---|---|
Vec<T> , String , ... |
i32 , char , ... |
[T; n] , (T1,T2,T3) , ... |
struct Car { color: Color, engine: Engine }
fn demo_ownership() {
let mut used_car: Car = Car { color: Color::Red,
engine: Engine::BrokenV8 };
let apartments = ApartmentBuilding::new();
references to data (&mut T
, &T
):
let my_home: &Home; // <-- an "immutable" borrow
let christine: &mut Car; // <-- a "mutable" borrow
my_home = &apartments[6]; // (read `mut` as "exclusive")
let neighbors_home = &apartments[5];
christine = &mut used_car;
christine.engine = Engine::VintageV8;
}
&
-reference types?Distinguish exclusive access from shared access
Enables safe, parallel API's
let christine = Car::new();
This is "Christine"
(apologies to Stephen King)
let read_only_borrow = &christine;
(apologies to Randall Munroe)
read_only_borrows[2] = &christine;
read_only_borrows[3] = &christine;
read_only_borrows[4] = &christine;
When inspectors are finished, we are left again with:
let mutable_borrow = &mut christine; // like taking keys ...
give_arnie(mutable_borrow); // ... and giving them to someone
read_only_borrows[2] = &christine;
let mutable_borrow = &mut christine;
read_only_borrows[3] = &christine;
// ⇒ CHAOS!
Ownership | T |
|
Exclusive access | &mut T |
("mutable") |
Shared access | &T |
("read-only") |
&mut
: can I borrow the car?fn borrow_the_car_1() {
let mut christine = Car::new();
{
let car_keys = &mut christine;
let arnie = invite_friend_over();
arnie.lend(car_keys);
} // end of scope for `arnie` and `car_keys`
christine.drive_to(work); // I still own the car!
}
But when her keys are elsewhere, I cannot drive christine
!
fn borrow_the_car_2() {
let mut christine = Car::new();
{
let car_keys = &mut christine;
let arnie = invite_friend_over();
arnie.lend(car_keys);
christine.drive_to(work); // <-- compile error
} // end of scope for `arnie` and `car_keys`
}
Possessing the keys, Arnie could take the car for a new paint job.
fn lend_1(arnie: &Arnie, k: &mut Car) { k.color = arnie.fav_color; }
Or lend keys to someone else (reborrowing) before paint job
fn lend_2(arnie: &Arnie, k: &mut Car) {
arnie.partner.lend(k); k.color = arnie.fav_color;
}
Owner loses capabilities attached to &mut
-borrows only temporarily (*)
(*): "Car keys" return guaranteed by Rust; sadly, not by physical world
let b = B::new();
let b = B::new();
let r1: &B = &b;
let r2: &B = &b;
(b
has lost write capability)
let mut b = B::new();
let w: &mut B = &mut b;
(b
has temporarily lost both read and write capabilities)
Box<B>
let a = Box::new(B::new());
a
(as owner) has both read and write capabilities
let a = Box::new(B::new());
let r_of_box: &Box<B> = &a; // (not directly a ref of B)
let r1: &B = &*a;
let r2: &B = &a; // <-- coercion!
a
retains read capabilities (has temporarily lost write)
let mut a = Box::new(B::new());
let w: &mut B = &mut a; // (again, coercion happening here)
a
has temporarily lost both read and write capabilities
Vec<B>
let mut a = Vec::new();
for i in 0..n { a.push(B::new()); }
...
a.push(B::new());
before | after | |
Vec<B>
let mut a = Vec::new();
for i in 0..n { a.push(B::new()); }
(a
has read and write capabilities)
let mut a = Vec::new();
for i in 0..n { a.push(B::new()); }
let r1 = &a[0..3];
let r2 = &a[7..n-4];
(a
has only read capability now; shares it with r1
and r2
)
&[..]
let mut a = Vec::new();
for i in 0..n { a.push(B::new()); }
let r1 = &a[0..7];
let r2 = &a[3..n-4];
Vec<B>
again(a
has read and write capabilities)
let w = &mut a[0..n];
(a
has no capabilities; w
now has read and write capability)
let (w1,w2) = a.split_at_mut(n-4);
(w1
and w2
share read and write capabilities for disjoint portions)
let rc1 = Rc::new(B::new());
let rc2 = rc1.clone(); // increments ref-count on heap-alloc'd value
(rc1
and rc2
each have read access; but neither can statically assume exclusive (mut
) access, nor can they provide &mut
borrows without assistance.)
RefCell<T>
: Dynamic Exclusivitylet b = Box::new(RefCell::new(B::new()));
let r1: &RefCell<B> = &b;
let r2: &RefCell<B> = &b;
RefCell<T>
: Dynamic Exclusivitylet b = Box::new(RefCell::new(B::new()));
let r1: &RefCell<B> = &b;
let r2: &RefCell<B> = &b;
let w = r2.borrow_mut(); // if successful, `w` acts like `&mut B`
// below panics if `w` still in scope
let w2 = b.borrow_mut();
Rc<RefCell<T>>
let rc1 = Rc::new(RefCell::new(B::new()));
let rc2 = rc1.clone(); // increments ref-count on heap-alloc'd value
Rc<RefCell<T>>
let rc1 = Rc::new(RefCell::new(B::new()));
let rc2 = rc1.clone();
let r1: &RefCell<B> = &rc1;
let r2: &RefCell<B> = &rc2; // (or even just `r1`)
Rc<RefCell<T>>
let rc1 = Rc::new(RefCell::new(B::new()));
let rc2 = rc1.clone();
let w = rc2.borrow_mut();
Rc<RefCell<T>>
have?Not much!
If you want to port an existing imperative algorithm with all sorts of sharing, you could try using Rc<RefCell<T>>
.
You then might spend much less time wrestling with Rust's type (+borrow) checker.
The point: Rc<RefCell<T>>
is nearly an anti-pattern. It limits static reasoning. You should avoid it if you can.
TypedArena<T>
Cow<T>
Rc<T>
vs Arc<T>
std::thread
dispatch
: OS X-specific "Grand Central Dispatch"
crossbeam
: Lock-Free Abstractions, Scoped "Must-be" Concurrency
rayon
: Scoped Fork-join "Maybe" Parallelism (inspired by Cilk)
(Only the first comes with Rust out of the box)
fn concurrent_web_fetch() -> Vec<::std::thread::JoinHandle<()>> {
use hyper::{self, Client};
use std::io::Read; // pulls in `chars` method
let sites = &["http://www.eff.org/", "http://rust-lang.org/",
"http://imgur.com", "http://mozilla.org"];
let mut handles = Vec::new();
for site_ref in sites {
let site = *site_ref;
let handle = ::std::thread::spawn(move || {
// block code put in closure: ~~~~~~~
let client = Client::new();
let res = client.get(site).send().unwrap();
assert_eq!(res.status, hyper::Ok);
let char_count = res.chars().count();
println!("site: {} chars: {}", site, char_count);
});
handles.push(handle);
}
return handles;
}
fn concurrent_gcd_fetch() -> Vec<::dispatch::Queue> {
use hyper::{self, Client};
use std::io::Read; // pulls in `chars` method
use dispatch::{Queue, QueueAttribute};
let sites = &["http://www.eff.org/", "http://rust-lang.org/",
"http://imgur.com", "http://mozilla.org"];
let mut queues = Vec::new();
for site_ref in sites {
let site = *site_ref;
let q = Queue::create("qcon2016", QueueAttribute::Serial);
q.async(move || {
let client = Client::new();
let res = client.get(site).send().unwrap();
assert_eq!(res.status, hyper::Ok);
let char_count = res.chars().count();
println!("site: {} chars: {}", site, char_count);
});
queues.push(q);
}
return queues;
}
lock-free data structures
scoped threading abstraction
upholds Rust's safety (data-race freedom) guarantees
crossbeam
MPSC benchmarkmean ns/msg (2 producers, 1 consumer; msg count 10e6; 1G heap)
108ns
|
98ns
|
53ns
|
461ns
|
192ns
|
Rust channel |
crossbeam MSQ
|
crossbeam SegQueue
|
Scala MSQ | Java ConcurrentLinkedQueue |
crossbeam
MPMC benchmarkmean ns/msg (2 producers, 2 consumers; msg count 10e6; 1G heap)
|
102ns
|
58ns
|
239ns
|
204ns
|
Rust channel (N/A) |
crossbeam MSQ
|
crossbeam SegQueue
|
Scala MSQ | Java ConcurrentLinkedQueue |
See "Lock-freedom without garbage collection" https://aturon.github.io/blog/2015/08/27/epoch/
std::thead
does not allow sharing stack-local data
fn std_thread_fail() {
let array: [u32; 3] = [1, 2, 3];
for i in &array {
::std::thread::spawn(|| {
println!("element: {}", i);
});
}
}
error: `array` does not live long enough
crossbeam
scoped threadingfn crossbeam_demo() {
let array = [1, 2, 3];
::crossbeam::scope(|scope| {
for i in &array {
scope.spawn(move || {
println!("element: {}", i);
});
}
});
}
::crossbeam::scope
enforces parent thread joins on all spawned children before returning
scope
: "must-be concurrency"Each scope.spawn(..)
invocation fires up a fresh thread
(Literally just a wrapper around std::thread
)
rayon
: "maybe parallelism"rayon
demo 1: map reducefn demo_map_reduce_seq(stores: &[Store], list: Groceries) -> u32 {
let total_price = stores.iter()
.map(|store| store.compute_price(&list))
.sum();
return total_price;
}
fn demo_map_reduce_par(stores: &[Store], list: Groceries) -> u32 {
let total_price = stores.par_iter()
.map(|store| store.compute_price(&list))
.sum();
return total_price;
}
the decision of whether or not to use parallel threads is made dynamically, based on whether idle cores are available
i.e., solely for offloading work, not for when concurrent operation is necessary for correctness
(uses work-stealing under the hood to distribute work among a fixed set of threads)
rayon
demo 2: quicksortfn quick_sort<T:PartialOrd+Send>(v: &mut [T]) {
if v.len() > 1 {
let mid = partition(v);
let (lo, hi) = v.split_at_mut(mid);
rayon::join(|| quick_sort(lo),
|| quick_sort(hi));
}
}
fn partition<T:PartialOrd+Send>(v: &mut [T]) -> usize {
// see https://en.wikipedia.org/wiki/
// Quicksort#Lomuto_partition_scheme
...
}
rayon
demo 3: buggy quicksortfn quick_sort<T:PartialOrd+Send>(v: &mut [T]) {
if v.len() > 1 {
let mid = partition(v);
let (lo, hi) = v.split_at_mut(mid);
rayon::join(|| quick_sort(lo),
|| quick_sort(hi));
}
}
fn quick_sort<T:PartialOrd+Send>(v: &mut [T]) {
if v.len() > 1 {
let mid = partition(v);
let (lo, hi) = v.split_at_mut(mid);
rayon::join(|| quick_sort(lo),
|| quick_sort(lo));
// ~~ data race!
}
}
(See blog post "Rayon: Data Parallelism in Rust" bit.ly/1IZcku4
)
3rd parties identify (and provide) new abstractions for concurrency and parallelism unanticipated in std lib.
Send
Sync
lifetime bounds
T: Send
means an instance of T
can be transferred between threads
(i.e. move or copied as appropriate)
T: Sync
means two threads can safely share a reference to an instance of T
T: Send
: T
can be transferred between threads
T: Sync
: two threads can share refs to a T
String
is Send
Vec<T>
is Send
(if T
is Send
)T: Sync
for Vec<T>: Send
?)Rc<T>
is not Send
(for any T
)Arc<T>
is Send
(if T
is Send
and Sync
)T:Send
for Arc<T>
?)&T
is Send
if T: Sync
&mut T
is Send
if T: Send
std::thread
is provided with std lib
But dispatch
, crossbeam
, and rayon
are 3rd party
(not to mention hyper
and a host of other crates used in this talk's construction)
What is Rust's code distribution story?
cargo
is really simple to use
cargo new -- create a project
cargo test -- run project's unit tests
cargo run -- run binaries associated with project
cargo publish -- push project up to crates.io
Edit the associated Cargo.toml
file to:
"What's this about crates.io
?"
Open-source crate distribution site
Has every version of every crate
Cargo adheres to semver
The use of Semantic Versioning in cargo
basically amounts to this:
Major versions (MAJOR.minor.patch) are free to break whatever they want.
New public API's can be added with minor versions updates (major.MINOR.patch), as long as they do not impose breaking changes.
In Rust, breaking changes includes data-structure representation changes.
Adding fields to structs (or variants to enums) can cause their memory representation to change.
Cargo invokes the Rust compiler in a way that salts the symbols exported by a compiled library.
This ends up allowing two distinct (major) versions of a library to be used simultaneously in the same program.
This is important when pulling in third party libraries.
cargo
generates a Cargo.lock
file that tracks the versions you built the project with
Intent: application (i.e. final) crates should check their Cargo.lock
into version control
However: library (i.e. intermediate) crates should not check their Cargo.lock
into version control.
Compiler ensures one cannot pass struct defined via X
version 2.x.y into function expecting X
version 1.m.n, or vice versa.
A : Graph Structure |
B : Token API |
|
C : Lexical Scanner |
D : GLL Parser |
P : Linked Program |
If you (*) follow the sem.ver. rules, then you do not usually have to think hard about those sorts of pictures.
"you" is really "you and all the crates you use"
cargo
is really simple to useRust to C
extern { ... }
and unsafe { ... }
C to Rust
#[no_mangle] extern "C" fn foo(...) { ... }
Ruby, Python, etc to Rust
https://github.com/wycats/rust-bridge
Mozilla (of course)
Skylight
MaidSafe
... others
Maidsafe is one example of this
Enabling sharing systems hacking knowledge with everyone
Programming in Rust has made me look at C++ code in a whole new light
www.rust-lang.org
Hack Without Fear