440 lines
No EOL
9.8 KiB
HTML
440 lines
No EOL
9.8 KiB
HTML
<!DOCTYPE html>
|
|
<html><head>
|
|
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
|
|
<title>Streampunk</title>
|
|
<meta charset="utf-8">
|
|
<script src="Streampunk_files/slides.js"></script>
|
|
<script>
|
|
|
|
if (window["location"] && window["location"]["hostname"] == "talks.golang.org") {
|
|
var _gaq = _gaq || [];
|
|
_gaq.push(["_setAccount", "UA-11222381-6"]);
|
|
_gaq.push(["b._setAccount", "UA-49880327-6"]);
|
|
window.trackPageview = function() {
|
|
_gaq.push(["_trackPageview", location.pathname+location.hash]);
|
|
_gaq.push(["b._trackPageview", location.pathname+location.hash]);
|
|
};
|
|
window.trackPageview();
|
|
window.trackEvent = function(category, action, opt_label, opt_value, opt_noninteraction) {
|
|
_gaq.push(["_trackEvent", category, action, opt_label, opt_value, opt_noninteraction]);
|
|
_gaq.push(["b._trackEvent", category, action, opt_label, opt_value, opt_noninteraction]);
|
|
};
|
|
}
|
|
</script>
|
|
<link href="Streampunk_files/light.css" type="text/css" rel="stylesheet"><meta content="width=1100,height=750" name="viewport"><meta content="yes" name="apple-mobile-web-app-capable"></head>
|
|
|
|
<body class="loaded" style="display: none">
|
|
|
|
<section class="slides layout-widescreen">
|
|
|
|
<article class="current">
|
|
<h1>Streampunk</h1>
|
|
<h3>A minimal, distributed Icecast cluster</h3>
|
|
<h3>25 January 2016</h3>
|
|
|
|
<div class="presenter">
|
|
|
|
</div>
|
|
|
|
</article>
|
|
|
|
|
|
|
|
<article class="next">
|
|
|
|
<h3>Streampunk</h3>
|
|
|
|
|
|
<p>
|
|
Streampunk.cc is a small, self-sustained organization providing Icecast streaming services to free radios.
|
|
</p>
|
|
|
|
<p class="link"><a href="http://streampunk.cc/" target="_blank">streampunk.cc/</a></p>
|
|
|
|
<p>
|
|
It is based on Autoradio, a software dedicated to these goals.
|
|
</p>
|
|
|
|
<p class="link"><a href="https://git.autistici.org/ale/autoradio" target="_blank">git.autistici.org/ale/autoradio</a></p>
|
|
|
|
</article>
|
|
|
|
|
|
|
|
<article class="far-next">
|
|
|
|
<h3>Why offer a streaming service?</h3>
|
|
|
|
<ul>
|
|
|
|
<li>communication is important, radios are nice, etc.</li>
|
|
|
|
<li>keeping up a single Icecast server is easy, keeping a reliable service is not</li>
|
|
|
|
<li>"aggregating" streams on a single platform makes economical sense</li>
|
|
|
|
</ul>
|
|
|
|
|
|
</article>
|
|
|
|
|
|
|
|
<article>
|
|
|
|
<h3>Why new software?</h3>
|
|
|
|
|
|
<p>
|
|
A few reasons to build this software:
|
|
</p>
|
|
|
|
|
|
<ul>
|
|
|
|
<li>educational purpose (coordination primitives on etcd, traffic control)</li>
|
|
|
|
<li>scaling / management issues with existing radios (<a href="http://ondarossa.info/" target="_blank">Ondarossa</a>)</li>
|
|
|
|
</ul>
|
|
|
|
|
|
<p>
|
|
Radios often need to deal with large traffic spikes, for example when covering
|
|
<br>
|
|
|
|
large demonstrations, and their utilization varies wildly.
|
|
</p>
|
|
|
|
|
|
<ul>
|
|
|
|
<li>keeping many machines always on is just too expensive; we want it to be easy to add new machines when we really need it</li>
|
|
|
|
<li>shit happens: the day of the big demo is the day your main Icecast server will go down</li>
|
|
|
|
</ul>
|
|
|
|
|
|
</article>
|
|
|
|
|
|
|
|
<article>
|
|
|
|
<h3>Alternatives</h3>
|
|
|
|
|
|
<p>
|
|
What to do when one machine isn't enough anymore:
|
|
</p>
|
|
|
|
|
|
|
|
<p>
|
|
<i>1)</i> standard Icecast relay setup
|
|
</p>
|
|
|
|
|
|
<ul>
|
|
|
|
<li>no mechanism to route clients</li>
|
|
|
|
<li>master is SPOF</li>
|
|
|
|
</ul>
|
|
|
|
|
|
<p>
|
|
<i>2)</i> <a href="http://giss.tv/" target="_blank">giss.tv</a> Icecast patches (single traffic-controlling master)
|
|
</p>
|
|
|
|
|
|
<ul>
|
|
|
|
<li>master redirects clients to least-loaded relay</li>
|
|
|
|
<li>master is SPOF</li>
|
|
|
|
</ul>
|
|
|
|
|
|
<p>
|
|
Single points of failure are annoying because they add maintenance
|
|
overhead. Prolonged master failures are bad because that's where all
|
|
the sources are connected, so they make the rest of the network useless.
|
|
</p>
|
|
|
|
|
|
|
|
</article>
|
|
|
|
|
|
|
|
<article>
|
|
|
|
<h3>Distributed systems that scale down</h3>
|
|
|
|
|
|
<p>
|
|
We think that there is a somewhat unexplored niche in distributed
|
|
systems in these times of big data: systems that are meant to <i>scale</i> <i>down</i> easily.
|
|
</p>
|
|
|
|
|
|
<ul>
|
|
|
|
<li>small, efficient (simplistic) codebase</li>
|
|
|
|
<li>designed to take advantage of tiny, low-power machines</li>
|
|
|
|
<li>all components of a "real" distributed system</li>
|
|
|
|
</ul>
|
|
|
|
|
|
<p>
|
|
This design can produce reliable services that are extremely cheap to run, both in terms of cost and administrator time.
|
|
</p>
|
|
|
|
|
|
|
|
</article>
|
|
|
|
|
|
|
|
<article>
|
|
|
|
<h3>Autoradio</h3>
|
|
|
|
|
|
<p>
|
|
<i>autoradio</i> can coordinate a bunch of machines into a
|
|
fault-tolerant Icecast cluster. It removes the single point of failure
|
|
by running a master-election protocol on top of a distributed database
|
|
(etcd).
|
|
</p>
|
|
|
|
|
|
|
|
<p>
|
|
We use a few simple distributed primitives:
|
|
</p>
|
|
|
|
|
|
<ul>
|
|
|
|
<li>presence broadcasts (with utilization stats)</li>
|
|
|
|
<li>master-election (using CAS and heartbeats)</li>
|
|
|
|
</ul>
|
|
|
|
|
|
<p>
|
|
The cluster tolerates failure of <i>N/2-1</i> nodes. It can
|
|
guarantee recovery from node failures (including the master) within the
|
|
time constant of the master-election protocol, usually a few seconds.
|
|
</p>
|
|
|
|
|
|
|
|
<p>
|
|
The service can easily scale horizontally by adding new machines with an automated procedure.
|
|
</p>
|
|
|
|
|
|
|
|
</article>
|
|
|
|
|
|
|
|
<article>
|
|
|
|
<h3>Autoradio : traffic flow</h3>
|
|
|
|
|
|
<p>
|
|
Traffic is controlled at the DNS and HTTP level. Sources are reverse
|
|
proxied to the master, while clients are redirected to the appropriate
|
|
relay. In more detail:
|
|
</p>
|
|
|
|
|
|
|
|
<p>
|
|
<i>1)</i> Client finds a server by using DNS. There is little control over this stage, client hopefully will get a working server.
|
|
</p>
|
|
|
|
|
|
|
|
<p>
|
|
<i>2)</i> Client makes a HTTP request for the stream URL. Autoradio
|
|
picks a server for the new connection, and sends back a redirect to a
|
|
special "passthrough" URL on the chosen server.
|
|
</p>
|
|
|
|
|
|
|
|
<p>
|
|
<i>3)</i> The client makes the final HTTP request on the desired
|
|
backend, autoradio on that machine simply proxies the connection to the
|
|
local Icecast daemon.
|
|
</p>
|
|
|
|
|
|
|
|
<p>
|
|
Problems: some very old clients do not understand HTTP redirects.
|
|
</p>
|
|
|
|
|
|
|
|
</article>
|
|
|
|
|
|
|
|
<article>
|
|
|
|
<h3>Autoradio : traffic control</h3>
|
|
|
|
|
|
<p>
|
|
Autoradio implements a tiny programmable traffic control engine,
|
|
with predictive estimates of backend utilization. What this means is
|
|
that it can:
|
|
</p>
|
|
|
|
|
|
<ul>
|
|
|
|
<li>prevent individual servers from overloading</li>
|
|
|
|
<li>mitigates thundering herd effects in case of high connection rates</li>
|
|
|
|
<li>distribute load somewhat equally to minimize impact of single server loss</li>
|
|
|
|
</ul>
|
|
|
|
|
|
<p>
|
|
The server selection algorithm can be configured, for example:
|
|
</p>
|
|
|
|
|
|
|
|
<div class="code"><pre>listeners_available,listeners_score,weighted</pre></div>
|
|
|
|
|
|
|
|
<p>
|
|
will distribute requests randomly, weighted by listener utilization,
|
|
and will block requests to servers with utilization greater than 100%.
|
|
</p>
|
|
|
|
|
|
|
|
</article>
|
|
|
|
|
|
|
|
<article>
|
|
|
|
<h3>Autoradio : transcoding</h3>
|
|
|
|
|
|
<p>
|
|
Operators can create <i>transcoded</i> streams, providing a version of a stream automatically re-encoded with different parameters (codec, bitrate).
|
|
</p>
|
|
|
|
|
|
|
|
<p>
|
|
Autoradio uses <a href="http://liquidsoap.fm/" target="_blank">liquidsoap</a>
|
|
to do this, running an instance for each transcoded stream, wrapped
|
|
within a master-election protocol to guarantee global uniqueness.
|
|
</p>
|
|
|
|
|
|
|
|
</article>
|
|
|
|
|
|
|
|
<article>
|
|
|
|
<h3>Meanwhile, clients</h3>
|
|
|
|
|
|
<p>
|
|
we discovered that many clients don't really know HTTP: they work with Icecast,
|
|
<br>
|
|
|
|
but not with a real HTTP server.
|
|
</p>
|
|
|
|
|
|
|
|
<p>
|
|
<code>butt</code>, we're looking at you
|
|
</p>
|
|
|
|
|
|
|
|
<p>
|
|
Icecast doesn't really speak HTTP, either.
|
|
</p>
|
|
|
|
|
|
|
|
</article>
|
|
|
|
|
|
|
|
<article>
|
|
|
|
<h3>Questions?</h3>
|
|
|
|
<div class="background">
|
|
<img src="Streampunk_files/streampunk.jpg">
|
|
</div>
|
|
|
|
|
|
</article>
|
|
|
|
|
|
|
|
<article>
|
|
<h3>Thank you</h3>
|
|
|
|
<div class="presenter">
|
|
<p class="link"><a href="mailto:ale@incal.net" target="_blank">ale@incal.net</a></p><p class="link"><a href="http://streampunk.cc/" target="_blank">http://streampunk.cc/</a></p>
|
|
</div>
|
|
|
|
</article>
|
|
|
|
<div id="prev-slide-area" class="slide-area"></div><div id="next-slide-area" class="slide-area"></div></section>
|
|
|
|
<div style="display: none;" id="help">
|
|
Use the left and right arrow keys or click the left and right
|
|
edges of the page to navigate between slides.<br>
|
|
(Press 'H' or navigate to hide this message.)
|
|
</div>
|
|
|
|
|
|
<script src="Streampunk_files/play.js"></script>
|
|
|
|
|
|
<script>
|
|
(function() {
|
|
|
|
if (window["location"] && window["location"]["hostname"] == "talks.golang.org") {
|
|
var ga = document.createElement("script"); ga.type = "text/javascript"; ga.async = true;
|
|
ga.src = ("https:" == document.location.protocol ? "https://ssl" : "http://www") + ".google-analytics.com/ga.js";
|
|
var s = document.getElementsByTagName("script")[0]; s.parentNode.insertBefore(ga, s);
|
|
}
|
|
})();
|
|
</script>
|
|
|
|
|
|
<div style="display: none;" id="ytCinemaMessage"></div><link href="Streampunk_files/css.css" type="text/css" rel="stylesheet"><link href="Streampunk_files/styles.css" type="text/css" rel="stylesheet"></body></html> |