Free segment script

DataparkSearch consists of two parts. The first part is indexing mechanism (indexer). Indexer walks over html hypertext references and stores found words and new references into database. The second part is web CGI front-end to provide search using data collected by indexer.

Major Features:

1. Support for http, https, ftp, nntp and news URL schemes.
2. htdb virtual URL scheme support for indexing SQL databases.
3. text/html, text/xml, text/plain, audio/mpeg (MP3) and image/gif mime types built-in support.
4. External parsers support for other document types.
5. Ability to index multilingual sites using content negotiation.
6. Searching all of the word forms using ispell affixes and dictionaries.
7. Fuzzy searching based on acronyms and abbreviations.
8. Stop-words, synonyms and acronyms lists.
9. Boolean query language support.
10. Neural network based Popularity Rank.
11. Results sorting by relevance, popularity rank, last modified time and by importance (a multiplication of relevance and popularity rank).
12. Various character sets support.
13. Accent insensitive search.
14. Phrase segmenting for Chinese, Japanese, Korean and Thai languages.
15. mod_dpsearch search module for Apache web server.
16. Internationalized Domain Names support.
17. The Summary Extraction Algorithm.

If you exit lftp when some jobs are not finished yet, lftp will move itself to nohup mode in background. The same happens when you have a real modem hangup or when you close an xterm.lftp has shell-like command syntax allowing you to launch several commands in parallel in background. It is also possible to group commands within () and execute them in background.

Major Features:

1. FTP and HTTP protocols support.
2. FTP (e.g. TIS FWTK) proxy support.
3. HTTP proxy support.
4. FTP over HTTP proxy support (hftp).
5. HTTPS and FTPS protocols support using GNU TLS or OpenSSL library.
6. Automatic OPIE/SKEY support in FTP protocol.
7. FXP transfers support (between two FTP servers, bypassing client machine).
8. FTP listings support: unix-style, NT, EPLF, OS/2, AS400, MacWebStar, MLSD.
9. Automatic reconnect and retry of an operation on retriable errors or timeout.
10. IPv6 support in both FTP and HTTP.
11. FISH protocol support. It uses ssh with no special program on server side.
12. SFTP protocol v3-v6 support.
13. HTTP/1.1 keep-alive support.
14. Partial WebDAV support.
15. Experimental support for SRV DNS records.
16. SOCKS support (configure option).
17. Modification time preservation (if server can report it).
18. `reget and `reput support.
19. Built-in mirror and reverse mirror (mirror -R).
20. Transfer rate throttling for each connection and for all connections in sum.
21. Limit for number of connections to the same site. Interruption of background transfer to do a foreground operation when the limit is reached.
22. Socket options tuning: buffer size and maximum segment size.
23. Job queueing.
24. Job execution at specified time.
25. Comprehensive scripting support.
26. URL recognition in most operations.
27. Readline library is used for input.
28. Context sensitive completion.
29. Bash-like prompt extensions.
30. Launching of local shell commands.
31. Suspend support.
32. Bookmarks.
33. Aliases.
34. Saving of last directory on each site. `cd to go there.
35. Copying of files between two servers, e.g. between FTP and HTTP.
36. Numerous settings which can be associated with a specific server or a group.
37. Paging of remote files using external program.
38. `zcat and `zmore support.
39. Help for commands.
40. Command history saved in a file.
41. Transfer status includes rate and ETA.
42. File listing cache.