Spotify Switches From Debian To Ubuntu

[jimbohale]

Right. I'm sure Spotify counted ALL their servers when they said 5000. Still, Spotify uses bittorrent to distribute the load across their users, so I still agree that it's odd that they need that many servers.

I don't think they use Bittorrent. Just because it's p2p doesn't mean it's bittorrent. Additionally, they have the p2p network just in case they need it. So, for example, if you listen to a song it attempts to download and cache it, and if someone else listens to the same song after you there is a chance that they will download portions of that file from you.

[My8th]

I don't think they use Bittorrent. Just because it's p2p doesn't mean it's bittorrent. Additionally, they have the p2p network just in case they need it. So, for example, if you listen to a song it attempts to download and cache it, and if someone else listens to the same song after you there is a chance that they will download portions of that file from you.

Spotify has stopped using P2P this year

[miquels]

Why do they need 5000 servers anyway? Netflix streams video, accounting for more than half the Internet traffic in the US at peak times, from only 50 servers or so. FreeBSD, just saying.

Apparently Netflix does "several terabits/sec" in the USA. Let's say 5 Tb/s. That's 5000 gigabit/s. So each server would be sending out 100 gigabit/s. That's really impressive for an Amazon VM!

In reality, Netflix has thousands of VMs in multiple Amazon EC2 zones. On top of that, they install local cache servers at ISPs around the world.

Mike.

[xeekei]

I don't think they use Bittorrent. Just because it's p2p doesn't mean it's bittorrent. Additionally, they have the p2p network just in case they need it. So, for example, if you listen to a song it attempts to download and cache it, and if someone else listens to the same song after you there is a chance that they will download portions of that file from you.

Why wouldn't they use bittorrent if it does exactly what they need? Why create their own thing? Especially since Spotify's founder and CEO, Daniel Ek, was briefly the CEO of the company behind ?Torrent in the past.

[eidolon]

Both. They were already using systemd, and I am talking about the fact that Debian offers no commercial support.

Via a cursory web search, I couldn't find clear mention that they had already deployed systemd across their Squeeze installations, only that they endorsed systemd as their preferred init system moving forward, but it's certainly possible. I know that systemd is included in Wheezy as a technology preview, but perhaps Spotify's engineers ported it to Squeeze.

It is apparent that they were already considering transitioning to Ubuntu at the time of Spotify's infrastructure team's comment to the bug report (https://news.ycombinator.com/item?id=7078301).

[jimbohale]

Why wouldn't they use bittorrent if it does exactly what they need? Why create their own thing? Especially since Spotify's founder and CEO, Daniel Ek, was briefly the CEO of the company behind ?Torrent in the past.

It's not exactly what they need. They need a way to transfer arbitrary chunks based on a certain song, which bittorrent technically provides but not really. A simpler, more cost effective, more reliable system would instead have a metadata server that would maintain who has which song and when it makes a lightweight request to the metadata server it would receive a list of peers who have that, and download chunks from those peers. It's similar to bittorrent but realistically so is every p2p protocol because that's the way it's done. If I can think it up in 20 seconds while writing this post I'm sure the engineers are spotify could think something up that would fit their needs much better than bittorrent (having the metadata servers or trackers being more lightweight and such since it doesn't need all of what bittorrent has).

Ultimately I know that spotify uses my bandwidth and I don't care. It doesn't use much (in fact it very rarely uses any at all).

[chrisb]

I don't really think these type of installs need LTS. Stable is old enough when it comes out, you get about 2-3 years then you get about another year to migrate away.

LTS is not just about stable software versions. One of the problems with using Debian stable in a business environment is that Debian does not do hardware enablement for stable releases. Imagine that a company procures a new model of servers that have some recent hardware that is not supported by Debian stable.. Now you have a problem. There are workarounds, like building your own install ISOs with back ported kernels, but then you have to assign an employee to do that work. And you have to assign an employee the job of tracking security updates for your back ported kernels. And back porting fixes for your installer ISO. So, it is all possible, but it is a hard sell against "let's just use Ubuntu and let them work about all of that stuff".

[stevenc]

Apparently Netflix does "several terabits/sec" in the USA. Let's say 5 Tb/s. That's 5000 gigabit/s. So each server would be sending out 100 gigabit/s. That's really impressive for an Amazon VM!

I stand corrected, they'd seem to have 1000 of those servers (OpenConnect appliances), not 50. They're not EC2 instances. They do have 10GbE interfaces (or multiple). And they do run FreeBSD

[ownagefool]

LTS is not just about stable software versions. One of the problems with using Debian stable in a business environment is that Debian does not do hardware enablement for stable releases. Imagine that a company procures a new model of servers that have some recent hardware that is not supported by Debian stable.. Now you have a problem. There are workarounds, like building your own install ISOs with back ported kernels, but then you have to assign an employee to do that work. And you have to assign an employee the job of tracking security updates for your back ported kernels. And back porting fixes for your installer ISO. So, it is all possible, but it is a hard sell against "let's just use Ubuntu and let them work about all of that stuff".

That's a good point but generally if you're going with a stable kernel, you should probably be going with stable hardware and it's not terribly hard to do this in a sever environment.

I dunno, I'm making a rather big assumption that their environment is resonably automated and uniform though, could be totally wrong.