Content delivery and distributed file sharing networks: Difference between revisions
imported>Howard C. Berkowitz (heading typo; bandwidth economics) |
mNo edit summary |
||
| (8 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
{{subpages}} | {{subpages}} | ||
There are a variety of technologies for increasing the efficiency of files and other content | {{TOC|right}} | ||
There are a variety of technologies for increasing the efficiency of access to files and other content needed by multiple users. Some of these technologies have come into conflict with laws and business concerns about [[intellectual property|intellectual property rights]], although it must be made clear that virtually all the conflicts involve the content being transferred, not the means of transfer. These techniques are an important part of [[convergence of communications]], not for services supporting active conversations, but for distribution of information as diverse as software updates and digital music. | |||
The means of transfer differ in the degree of distribution, ranging from serverless [[peer-to-peer]] (P2P), to peer-to-peer services with servers playing the role of broker to find desired content, to [[content distribution network]]s (CDN) with distributed servers owned by the content provider or a CDN contracted to it. Distributed information sharing is yet another paradigm that addresses some of the same issues as traditional [[file transfer]], and evolving [[distributed file system]]s such as the [[Network File System]] (NFS). In distributed information sharing, there is far less of an emphasis on client/server than in other protocols. It is more similar to the [[Network News Transfer Protocol]] (NTTP). | The means of transfer differ in the degree of distribution, ranging from serverless [[peer-to-peer]] (P2P), to peer-to-peer services with servers playing the role of broker to find desired content, to [[content distribution network]]s (CDN) with distributed servers owned by the content provider or a CDN contracted to it. Distributed information sharing is yet another paradigm that addresses some of the same issues as traditional [[file transfer]], and evolving [[distributed file system]]s such as the [[Network File System]] (NFS). In distributed information sharing, there is far less of an emphasis on client/server than in other protocols. It is more similar to the [[Network News Transfer Protocol]] (NTTP). | ||
| Line 8: | Line 9: | ||
Needless to say, there was a conflict between the copyright holders of entertainment content, and music users who believed music was overpriced. A portion of participants in P2P networks indeed were copyright holders, who believed that the ability to sample tracks would get new customers to buy albums or attend live shows. The legality and morality of these practices are beyond the scope of this article, other than to notice they created much network demand and, indeed, new technologies. It is worth noting that several P2P networks have closed, or had major sites refuse to download their software. | Needless to say, there was a conflict between the copyright holders of entertainment content, and music users who believed music was overpriced. A portion of participants in P2P networks indeed were copyright holders, who believed that the ability to sample tracks would get new customers to buy albums or attend live shows. The legality and morality of these practices are beyond the scope of this article, other than to notice they created much network demand and, indeed, new technologies. It is worth noting that several P2P networks have closed, or had major sites refuse to download their software. | ||
All open implementations that can distribute files can be used to distribute both legal and illegal information, just as a photocopier can copy unauthorized material. At the same time, some large entertainment corporations have been extremely aggressive in legal attack, to the extent of creating [[chilling effect]] on legitimate users, demanding expensive and possibly intrusive technologies to be implemented by [[Internet Service | All open implementations that can distribute files can be used to distribute both legal and illegal information, just as a photocopier can copy unauthorized material. At the same time, some large entertainment corporations have been extremely aggressive in legal attack, to the extent of creating [[chilling effect]] on legitimate users, demanding expensive and possibly intrusive technologies to be implemented by [[Internet Service Provider]]s, and even suggesting that digital rights enforcement be built into standard [[integrated circuit]]s that might be used for a purpose totally different than entertainment. | ||
===Bandwidth economics=== | ===Bandwidth economics=== | ||
College network administrators, in particular, found music downloads to take seemingly infinite amounts of bandwidth, interfering with their academic and research methods. Some administrators simply blocked ports used for file sharing, or content detected as [[multimedia]] (e.g., [[MP3]]). This was an issue of protecting resources, which handily countered arguments about academic freedom and censorship. | College network administrators, in particular, found music downloads to take seemingly infinite amounts of bandwidth, interfering with their academic and research methods. Some administrators simply blocked ports used for file sharing, or content detected as [[multimedia]] (e.g., [[MP3]]). This was an issue of protecting resources, which handily countered arguments about academic freedom and censorship. | ||
ISPs also may limit residential bandwidth on P2P ports, and sometimes have produced data showing that a small portion of the users are consuming a great deal of the bandwidth; quota systems or pricing on bandwidth overages may sometimes be needed. | ISPs also may limit residential bandwidth on P2P ports, and sometimes have produced data showing that a small portion of the users are consuming a great deal of the bandwidth; quota systems or pricing on bandwidth overages may sometimes be needed. Contrary to some allegations, [[network neutrality]] is not intended to interfere with bandwidth-sensitive pricing; its focus is on content provider-sensitive pricing. | ||
Of course, academic institutions tend to have large numbers of computer-literate individual with time and resources, so the situation began to resemble the military minuet of [[electronic warfare]]: electronic attack, countermeasure, and response by counter-countermeasures. It is noteworthy that the military has generalized to the concept of | Of course, academic institutions tend to have large numbers of computer-literate individual with time and resources, so the situation began to resemble the military minuet of [[electronic warfare]]: electronic attack, countermeasure, and response by counter-countermeasures. It is noteworthy that the military has generalized to the concept of information operations, which includes electronic warfare, but also attack, countermeasure, and counterattack on computers and computer networks. | ||
P2P users, especially with questionable legal positions, might encrypt content, defeating MP3 detection, and send it to a nonstandard port, or just try nonstandard (or dynamically changing) ports until one was blocked. | P2P users, especially with questionable legal positions, might encrypt content, defeating MP3 detection, and send it to a nonstandard port, or just try nonstandard (or dynamically changing) ports until one was blocked. | ||
===Technological assistance=== | |||
In the [[Internet Engineering Task Force]], the Application-Layer Traffic Optimization (alto) Working Group has been formed to "will design and specify an Application-Layer Traffic Optimization (ALTO) service that will <blockquote>provide applications with | |||
information to perform better-than-random initial peer selection. | |||
ALTO services may take different approaches at balancing factors | |||
such as maximum bandwidth, minimum cross-domain traffic, lowest cost to the | |||
user, etc. The WG will consider the needs of BitTorrent, tracker-less | |||
P2P, and other applications, such as content delivery networks (CDN) | |||
and mirror selection. <ref name=ALTO-Charter>{{citation | |||
| url = https://datatracker.ietf.org/wg/alto/charter/ | |||
| title = Application-Layer Traffic Optimization (alto) | |||
| publisher = [[Internet Engineering Task Force]]}}</ref></blockquote> | |||
Alto is ''not'' initially concerned with the P2P protocols themselves, but communicating with ALTO services to inform clients of types of information P2P applications may need for optimizing their choices. | |||
==Content delivery networks (CDN)== | ==Content delivery networks (CDN)== | ||
| Line 30: | Line 44: | ||
===BitTorrent=== | ===BitTorrent=== | ||
BitTorrent is a hybrid between pure peer-to-peer, and varying levels of server involvement. The servers are more concerned with directory functions and managing content flows that actually holding the content. BitTorrent seems to have been much more successful than other P2P companies in establishing working relationships with the entertainment industry, avoiding the adversarial relationships that closed down other P2P systems. When the protocol first deployed, it was at end user quality levels. Evolved versions can meet enterprise quality needs. There is also a content distribution service, based on BitTorrent technology but as opposed to P2P, with contractual SLAs. CacheLogic runs this service. BitTorrent claims that its traffic is 35% of the total Internet traffic, although this has not been independently confirmed | [[Image:File-sharing-via-BitTorrent.png|thumb|left|300px|Symbolic representation of BitTorrent]] | ||
{{main|BitTorrent}} | |||
BitTorrent is a hybrid between pure peer-to-peer, and varying levels of server involvement. The servers are more concerned with directory functions and managing content flows that actually holding the content. BitTorrent seems to have been much more successful than other P2P companies in establishing working relationships with the entertainment industry, avoiding the adversarial relationships that closed down other P2P systems. When the protocol first deployed, it was at end user quality levels. Evolved versions can meet enterprise quality needs. There is also a content distribution service, based on BitTorrent technology but as opposed to P2P, with contractual SLAs. [[CacheLogic]] runs this commercial service. BitTorrent claims that its traffic is 35% of the total Internet traffic, although this has not been independently confirmed. | |||
The original BitTorrent client, written in the [[Python programming language]], had its source code released under the BitTorrent Open Source License, which is a modified version of the [[Jabber]] Open Source License. There are numerous compatible clients, written in a variety of programming languages, and running on a variety of computing platforms. | |||
Where several P2P vendors have gone out of business, typically due to huge legal problems with pirated information, BitTorrent seems to have managed to stay more mainstream. BitTorrent distributes BitTorrent's own updates, as well as OpenOffice.org and most LINUX distributions. Free and Open Source conferences have distributed video of their conferences via BitTorrent. NASA uses it for large image files. | Where several P2P vendors have gone out of business, typically due to huge legal problems with pirated information, BitTorrent seems to have managed to stay more mainstream. BitTorrent distributes BitTorrent's own updates, as well as OpenOffice.org and most LINUX distributions. Free and Open Source conferences have distributed video of their conferences via BitTorrent. NASA uses it for large image files. | ||
Bram Cohen, the inventor of BitTorrent, and the movie industry executed an agreement in 2005. It required that the bittorrent.com website not have any linked to unlicensed movie content. Of course, this had no effect on other servers not under Cohen's control, but it appears there is a mutual attempt to work out compromises between this P2P operator and the entertainment industry than with several other services. | Bram Cohen, the inventor of BitTorrent, and the movie industry executed an agreement in 2005. It required that the bittorrent.com website not have any linked to unlicensed movie content. Of course, this had no effect on other servers not under Cohen's control, but it appears there is a mutual attempt to work out compromises between this P2P operator and the entertainment industry than with several other services. | ||
==References== | ==References== | ||
{{reflist}} | {{reflist}}[[Category:Suggestion Bot Tag]] | ||
Revision as of 16:00, 1 August 2024
There are a variety of technologies for increasing the efficiency of access to files and other content needed by multiple users. Some of these technologies have come into conflict with laws and business concerns about intellectual property rights, although it must be made clear that virtually all the conflicts involve the content being transferred, not the means of transfer. These techniques are an important part of convergence of communications, not for services supporting active conversations, but for distribution of information as diverse as software updates and digital music.
The means of transfer differ in the degree of distribution, ranging from serverless peer-to-peer (P2P), to peer-to-peer services with servers playing the role of broker to find desired content, to content distribution networks (CDN) with distributed servers owned by the content provider or a CDN contracted to it. Distributed information sharing is yet another paradigm that addresses some of the same issues as traditional file transfer, and evolving distributed file systems such as the Network File System (NFS). In distributed information sharing, there is far less of an emphasis on client/server than in other protocols. It is more similar to the Network News Transfer Protocol (NTTP).
Legal, ethical, and business issues
There are distinct security problems with some of the distributed models, and, indeed, some were shut down under legal pressure. The legal pressure came mostly from the entertainment industry, not amused with technology that enabled easy, royalty-free transfer of copyrighted music.
Needless to say, there was a conflict between the copyright holders of entertainment content, and music users who believed music was overpriced. A portion of participants in P2P networks indeed were copyright holders, who believed that the ability to sample tracks would get new customers to buy albums or attend live shows. The legality and morality of these practices are beyond the scope of this article, other than to notice they created much network demand and, indeed, new technologies. It is worth noting that several P2P networks have closed, or had major sites refuse to download their software.
All open implementations that can distribute files can be used to distribute both legal and illegal information, just as a photocopier can copy unauthorized material. At the same time, some large entertainment corporations have been extremely aggressive in legal attack, to the extent of creating chilling effect on legitimate users, demanding expensive and possibly intrusive technologies to be implemented by Internet Service Providers, and even suggesting that digital rights enforcement be built into standard integrated circuits that might be used for a purpose totally different than entertainment.
Bandwidth economics
College network administrators, in particular, found music downloads to take seemingly infinite amounts of bandwidth, interfering with their academic and research methods. Some administrators simply blocked ports used for file sharing, or content detected as multimedia (e.g., MP3). This was an issue of protecting resources, which handily countered arguments about academic freedom and censorship.
ISPs also may limit residential bandwidth on P2P ports, and sometimes have produced data showing that a small portion of the users are consuming a great deal of the bandwidth; quota systems or pricing on bandwidth overages may sometimes be needed. Contrary to some allegations, network neutrality is not intended to interfere with bandwidth-sensitive pricing; its focus is on content provider-sensitive pricing.
Of course, academic institutions tend to have large numbers of computer-literate individual with time and resources, so the situation began to resemble the military minuet of electronic warfare: electronic attack, countermeasure, and response by counter-countermeasures. It is noteworthy that the military has generalized to the concept of information operations, which includes electronic warfare, but also attack, countermeasure, and counterattack on computers and computer networks.
P2P users, especially with questionable legal positions, might encrypt content, defeating MP3 detection, and send it to a nonstandard port, or just try nonstandard (or dynamically changing) ports until one was blocked.
Technological assistance
In the Internet Engineering Task Force, the Application-Layer Traffic Optimization (alto) Working Group has been formed to "will design and specify an Application-Layer Traffic Optimization (ALTO) service that will
provide applications with
information to perform better-than-random initial peer selection. ALTO services may take different approaches at balancing factors such as maximum bandwidth, minimum cross-domain traffic, lowest cost to the user, etc. The WG will consider the needs of BitTorrent, tracker-less P2P, and other applications, such as content delivery networks (CDN)
and mirror selection. [1]
Alto is not initially concerned with the P2P protocols themselves, but communicating with ALTO services to inform clients of types of information P2P applications may need for optimizing their choices.
Content delivery networks (CDN)
Akamai pioneered content distribuion, using a push technology with content providers as their customers. They pushed the content to local caches at ISPs, which could deliver content to end users with less upstream bandwidth than the ISPs had previously needed, as well as incentives to the ISPs.
They have evolved their service to provide various edge accelerators, some intended as products and some as Akamai-run services. Time-Warner took over content distribution for CNN after Time-Warner bought the news company, and distributes Time-Warner content over its own CDN.
P2P with assistance in finding information
There are a wide range of P2P services whose relationships are established by personal contact and social networks. Their content may be legal or illegal to transfer.
Still, finding content can be a major challenge without purpose-built assistance. While it has shut down, Napster was the first widely available "peer to peer" service. It was not true point-to-point as was the original NetNews, because it did have servers to help mediate content exchanges, although the actual exchange was P2P.
Kazaa
Kazaa also is a system with "matchmaking" servers and direct P2P transfer between user machines. As with other programs, it had its assortment of legal problems with copyright. Helping Kazaa was a ruling from a Dutch court that it was, essentially, a carrier not responsible for the acts of its users.
BitTorrent
BitTorrent is a hybrid between pure peer-to-peer, and varying levels of server involvement. The servers are more concerned with directory functions and managing content flows that actually holding the content. BitTorrent seems to have been much more successful than other P2P companies in establishing working relationships with the entertainment industry, avoiding the adversarial relationships that closed down other P2P systems. When the protocol first deployed, it was at end user quality levels. Evolved versions can meet enterprise quality needs. There is also a content distribution service, based on BitTorrent technology but as opposed to P2P, with contractual SLAs. CacheLogic runs this commercial service. BitTorrent claims that its traffic is 35% of the total Internet traffic, although this has not been independently confirmed.
The original BitTorrent client, written in the Python programming language, had its source code released under the BitTorrent Open Source License, which is a modified version of the Jabber Open Source License. There are numerous compatible clients, written in a variety of programming languages, and running on a variety of computing platforms.
Where several P2P vendors have gone out of business, typically due to huge legal problems with pirated information, BitTorrent seems to have managed to stay more mainstream. BitTorrent distributes BitTorrent's own updates, as well as OpenOffice.org and most LINUX distributions. Free and Open Source conferences have distributed video of their conferences via BitTorrent. NASA uses it for large image files.
Bram Cohen, the inventor of BitTorrent, and the movie industry executed an agreement in 2005. It required that the bittorrent.com website not have any linked to unlicensed movie content. Of course, this had no effect on other servers not under Cohen's control, but it appears there is a mutual attempt to work out compromises between this P2P operator and the entertainment industry than with several other services.