Just make sure that the file name has “en” in the title, which indicates that the content is in English. You can also download a version without pictures (it’s called “wikipedia_en_all_nopic_2022-01.zim”) or smaller files with articles in certain topics, such as medicine, or football, or climate change. On the site, it’s called “wikipedia_en_all_maxi_2022-05.zim” and it’s 89GB. Clicked the 46GB file that contains all of English Wikipedia.How to do it Download with Kiwix onto computer: So here are some methods for downloading Wikipedia with as little pain as possible using tools from the nonprofit Kiwix. In addition, some downloaded files are set up for technical analysis - not pleasure reading. Like the rest of Wikipedia, many of the tools for downloading are maintained by volunteers, which is very generous and kind but also can mean that certain tools don’t receive the maintenance they need. Here is the coordination page.Wikipedia routinely makes a dump of its databases available publicly for free, and it explains a few different ways you can download its contents. One of the problem is that even on Gutenberg, we don't have all the most important books of the French litterature. Generate zimwriterfs-friendly folder of static HTML files based on templates and list of books.Generate a static folder repository of all ePUB files.Download the books based on filters (formats, languages).Query the database to reflect filters and get list of books.Loop through folder/files and parse RDF.Git clone git://.net/p/kiwix/other kiwix-other Sudo apt-get install libzim-dev liblzma-dev libmagic-dev autoconf automake The best Goobuntu packaged option seems to be: If you can somehow filter which books to fetch (language-only, book-range), that will be convenient So a on-disk-caching, robots-obeying url-retriever needs to be made/reused. So a caching fetch-by-url seems more convenient, the rdf-file contains the timestamp, which could be compared so updates to a book will be caught. To get epub+text+html, you'll need both rsync-trees, which seems quite inconvenient. If I cd gutenberg-generated, there is stuff like:
Rsync -av -del /var/www/gutenberg-generated Gutenberg supports rsync ( rsync -av -del /var/That was source, the generated data: Wget works, contains 30k directories with each an rdf-file: every directory has 1 file with the rdf-description of one book.Įmmanuel suggests the scraper should download everything into one dir, then converting the data into an output dir, then zim-ifying that directory.
Work done by didier chez and cniekel chez
0 kommentar(er)
