Oct 16, 2024, 09:37 PM
What's the best/fastest way to skim through or do a CTRL+F on a 60GB .txt or .json file?
|
Skimming through a large file
by lnonymous - Wednesday October 16, 2024 at 09:37 PM
|
|
Oct 16, 2024, 09:37 PM
What's the best/fastest way to skim through or do a CTRL+F on a 60GB .txt or .json file?
Oct 16, 2024, 11:30 PM
When having a file this big, your regular options are gonna struggle. You can't open a file this big in regular software like Excel or notepad because you (I doubt) don't have 60+GB of RAM.
You can create a simple python script that opens the file, reads it line by line, then prints any matches. If you want to go the proper way, you need to import your file in an SQL database. MySQL workbench has everything you need to get going. Databases engines are incredibly fast for going through large amounts of data like you have.
Oct 17, 2024, 07:20 AM
Oct 17, 2024, 09:09 AM
ripgrep if you can figure out how to get that running, you can find it on github
Oct 17, 2024, 02:50 PM
grep that shitttttt
Oct 17, 2024, 10:18 PM
bash: cat file.txt | grep "search term"
windows powershell: get-content file.txt | findstr -i "search term" Although if it is very large and you have some programming skill, write a parser in C, or just search using regular expressions in chunks of the file. This forum account is currently banned. Ban Length: Permanent (N/A Remaining)
Ban Reason: Self-Ban | Retired |http://c66go4clkqodr7tdjfu76jztjs7w7d3fajdeypxn73v4ju3dt7g5yyyd.onion/Forum-Ban-Appeals if you wish to be unbanned in the future.
Oct 18, 2024, 11:30 AM
REgular expressions man
"Universal appeal is poison masquerading as medicine. Horror is not meant to be universal. It's meant to be personal, private, animal"
Oct 19, 2024, 01:59 AM
|
|
« Next Oldest | Next Newest »
|
| Possibly Related Threads… | |||||
| Thread | Author | Replies | Views | Last Post | |
| NEW USERS READ - how to avoid malware on the forum | 104 | 12,857 |
6 hours ago Last Post: Elowyn |
||
| A collection of deepweb sites [2025] | 112 | 3,429 |
Yesterday, 07:57 PM Last Post: IsItReal |
||
| Hacking forums and their links 2026 | 1 | 274 |
Yesterday, 05:01 PM Last Post: phas3lock |
||
| What is your most efficent way to gain initial access? | 1 | 202 |
Yesterday, 04:47 PM Last Post: phas3lock |
||
| FREE 3 UNCENSORED HACKING LLM | 69 | 2,015 |
Yesterday, 04:37 PM Last Post: phas3lock |
||