BitTorrent is a peer-to-peer protocol for distributing large files over the internet, offering high performance downloads without the need to buy/rent a fast and expensive server environment.
Note that BitTorrent over a dialup connection will likely be an unsatisfactory experience. For one thing, BT is particularly apt for larger files; for another, because of the multiple server connections in use, the overhead for the connections would end up using a substantial portion of the available bandwidth.
Support for BitTorrent was originally included in Opera since version 8.02 Technical Preview 1. It was removed for the official 8.02 release, but resurfaced as a supported feature with version 9.
BitTorrent behaves similarly to an FTP or HTTP file transfer, with a few visible differences. The link to start a BitTorrent transfer points to a .torrent file. When Opera receives one of these files, it reads it, then informs the user that a BitTorrent is about to start. Given the OK to proceed, it starts making requests for the data from the available seeds and peers, and also notifies the tracker that it is accepting connections.
At this point, the Transfer window displays the status of the file. In the table of entries, this looks just like a normal transfer except that the speed column shows an upload speed alongside the download speed. The details for a BitTorrent entry likewise shows an upload transfer size alongsize the download transfer size, and also two dynamic lines of detail unique to BitTorrent: Connections which gives the number of seeds and peers available for transfer, and Active transfers which shows the number of inbound and outbound connections where data is currently being exchanged.
The prompt displayed by Opera to accept the BitTorrent transfer also displays which port it intends to open up (by default 18768; you might choose to change this to 6881, which is the standard port). It also provides a Test button so you can verify that this port is available to outside clients -- that is, not blocked by the NAT in your router or by your firewall. While it is possible to run BitTorrent transfers strictly via outbound requests (data can be uploaded over such links, if the client you've connected to needs parts of the file you've already received), you will get better upload speeds by opening up to inbound requests; and BitTorrent is designed to reward people who upload more with faster downloads.
Additionally, the prompt dialog has a Settings button, which opens another dialog allowing you to control the bandwidth allocated to BT transfers. Typically you'll want the download speed to be unlimited (and limiting the download speed is difficult anyway, essentially implemented by dropping packets which, in theory, should cause the source to back off a bit). You may need to limit upload speed, particularly if you're on an ADSL connection with a fairly slow upload connection. (In my very limited experience, even with the upload speed set relatively large, e.g. 20 KB/sec, the actual bandwidth used is much smaller.)
As of version 9, these settings are not accessible via the Preferences dialog; however, they are exposed in the Preferences Editor (opera:config), in a section labelled BitTorrent.
When you have completed your download, the Transfers dialog will change the 'Progress' display to Sharing files. At this point, you have become a Seed, and Opera will continue accepting requests for data from Peers. Since BitTorrent is exactly about providing maximum access to data, you should, whenever possible, allow this to continue. How long depends on the BT community you're interacting with; one rule-of-thumb is to keep your copy of the file available for 24 hours.