Our Logo  

Distributed Systems Lab 2003


Home Page
News & Feedback
Lab Organization
Lab Environment
Task Description
Overview
Lab 1
Lab 2
Lab 3
Lab 4
Certification Authority
WWW Tutorial
HTTP Tutorial
FAQ
Downloads

ShareMe - The DSG Files Sharing System


In this year's distributed systems lab, your task is to develop a file sharing system. You probably know many file sharing applications, e.g., Gnutella, FreeNet, Kazaa and many, many more. On this page you find an overview description of the whole system and the tasks you have to implement. A detailed description of every implementation task can be found in the corresponding task descriptions ([ 1 ], [ 2 ], [ 3 ], [ 4 ]).

In the lab, our goal is not to develop yet another file sharing system but to use file sharing as a sample scenario to explore the basics of developing distributed systems. It is not our intention to develop a perfect vanilla peer-to-peer file sharing system!

The ShareMe system is designed as a peer-to-peer system in which every peer (host, computer) acts in the roles of both a client and a server. In other words, every peer can locate and download files as well as share its own files with others. In this overview, we are going to dicuss the ShareMe system from a high-level point of view. It shall give you an idea of the basic structure and functionality of the system; the details are described in the task descriptions for the four lab examples.

Let's start right at the beginning; the only knowledge we have so far is that there is a number of computers connected via the network (i.e., the computers in the DSG lab). If you are implementing the lab at home, the computers are mimicked by several ShareMe instances on your local machine.

Hint: Make sure that your system works properly in the DSG lab environment before submitting it! The evaluation of your implementation will exclusively be done based on the lab environment.

Lab 1: Multicast communication

Having started the application, the first task of the ShareMe system is to locate the other ShareMe peers currently available on the network. Furthermore, it has to make itself known to the other peers (i.e., tell them that it is up and running now and can be used to share files with). We use multicast messages to keep track of all the other peers: a peer sends a multicast message every n seconds announcing that it is currently online ('IAmAlive' message). All other available peers do the same, thus our newly started system must listen to the 'IAmAlive' messages of the other peers. Using these messages your peer can maintain a list of all currently available servers. This list will be refreshed periodically as a result of subsequent 'IAmAlive' messages. If a server does not keep sending 'IAmAlive' messages, it is removed from the list of active hosts after a given timeout. Since all peers listen for 'IAmAlive' messages, new hosts are automatically detected and inactive hosts are automatically removed from the list.

What we gained in this first step is that every peer is able to maintain a list of other peers available in the system and this list is updated continuously. The remaining part of the lab 1 assignment is to provide a mechanism to shut down a peer. For this purpose a special shutdown message is used.

Figure 1 . Exchange of multicast 'IAmAlive' messages.

The essence of the first lab assignment is also displayed in Figure 1 . You see a total of four peers where peer #3 is currently not running. When node #1 gets started, it issues an 'IAmAlive' multicast message to announce its availability to the other peers out on the network (Step 1). The nodes #2 and #4 receive the message and add node #1 to the list of currently active peers. Of course, they follow the same procedure and regularly send multicast 'IAmAlive' messages on their own (Step 2). The above figure only shows the 'IAmAlive' messages of node #2 and #4 which are received by node #1; in reality, the (multicast) messages of nodes #2 and #4 are also received by all other nodes in the multicast group (i.e., all ShareMe peers). Since node #3 is down, it does not receive or send anything. After at most n seconds, node #1 receives the 'IAmAlive' messages of node #2 and #4 and, in turn, adds them to its list of available peers. This procedure is repeated periodically to keep the list of peers at all nodes up-to-date (with a time-window of about n seconds), i.e., to detect new peers as well as peers that went offline.

Lab 2: Searching for files

Based on the underlying infrastructure developed in lab 1, we now have to be able to search for files. For this purpose we use RMI communication. The 'IAmAlive' messages introduced in the previous lab assignment contain a lot of information about a particular ShareMe server including how and where to contact the server using RMI. Thus to search for a file we are interested in, we iterate through the list of currently active peers and contact them one by one sending our search request. Depending on the response of a server, we remember whether the server provides the requested file or not and proceed to the next server in the list. Eventually this leaves us with a list of servers from which we can obtain the requested files.

The next step is to implement the server part, i.e., add the functionality to respond to search requests of other peers. This, first of all, requires to have a list of files which can be shared with other peers. Secondly, the server has to locate the files in this list which match a search request. Finally, a search response is created and returned.

Figure 2 . Issuing search requests and responding with search results using RMI communication.

Figure 2 depicts what happens when a search request is issued. First the search request is created (Step 1). In the next step it is sent to all ShareMe peers currently online (peers #2 and #4 in our example) (Step 2). Finally, the peers process the request, respond and indicate whether they provide the requested file or not (Step 3).

Lab 3: Making searches secure

Since we have to make sure that nobody is fooling around and abusing our system (e.g., to create fake messages and flood the system with requests), we require search requests and responses to be digitally signed. A security infrastructure is used to achieve this requirement.

We use asymmetric cryptography to create digital signatures and a certification authority to distribute public keys. This is not a full-fledged security infrastructure (e.g., we ignore certificates!). The effort to integrate a real security infrastructure is too large for our purposes. Thus we restrict ourselves to public/private key pairs and signing of requests and responses. The certification authority is only used to distribute public keys and is implicitly trusted. Communication with the certification authority is done using CORBA.

Security influences our system in two places: in the role of a client, the peer has to sign requests and verify the digital signatures of signed responses. In the role of the server, a peer has to verify the incoming search requests and sign its responses, i.e., the same operations in the reverse order.

Figure 3 . Exchangine secure search requests and responses using digital signatures.

In Figure 3 we see what happens with a search request on the server side of the ShareMe system. First the request is created by the remote client (peer #2) (Step1) and digitally signed. Then the search request is sent to peer #1 where it is received (Step 2) and the signature of the request is verified (Step 3). Peer #1 then searches its list of shareable files and creates the appropriate response for the search request (Step 4). Finally the response is signed and returned to the requesting client (Step 5). The client then has to verify the digital signature of the incoming response using the certification authority (not shown in the figure).

Lab 4: Adding an HTML user interface

Now we are almost done. The only remaining step is to display the search result. This basically is a list of servers which provide the requested file for download. The user can then start the download from the server of her choice. Since we are not overly concerned about the 'best' user interface and want to have a platform-independent solution, we use a Web browser as our user interface of choice. This means that every peer also implements a small Web server which allows the user to submit search requests, view the search results and start the download via HTTP. Embedded into the scenario described so far, this means that a search request is processed in the following steps (as also shown in Figure 4 ).

Figure 4 . Searching and downloading files using an HTTP server and a Web browser.

The process starts when the user enters a search request in an HTML form displayed in the browser and submits the form (Step 1). The Web server must process the form's data and creates a search request object (Step2). All available ShareMe peers are queried and their responses are collected (just as what happened in lab 2 and 3) (Step 3). The search result, i.e., the list of all servers providing the requested file, is displayed in the browser (Step 4). Eventually, the user clicks a link for one of the servers (#2 in our example) and starts an HTTP download (Step 5). This means that the ShareMe peers' Web servers also have to understand requests that result in downloads of files.

From a communications point of view, the ShareMe system has several layers as indicated in Figure 5 . The list of currently online peers is managed using multicast communication. Search requests and responses are exchanged using RMI. Security information is exchanged with the certification authority via CORBA and the actual download process utilizes HTTP.

Figure 5 . The four layers of communication in the ShareMe system.

Having read this high-level overview you should have developed a basic understanding of how the ShareMe system works. Detailed descriptions of the four lab assignments can be found in their appropriate sections [ lab 1 ], [ lab 2 ], [ lab 3 ], [ lab 4 ].

Before you start implementing the system make sure you understand the organizational constraints as described in the organization document . If you run into problems implementing the ShareMe system, consult the information sources listed in the working mode document for support.

Good luck, the DSG Lab Team.

Powered by MyXML Last update on: 2003-03-13
© 2001 Distributed Systems Group