Advertisement

programmable / scriptable open source large file text editor - anyone know any?

Started by July 15, 2015 05:41 AM
15 comments, last by TheChubu 9 years, 2 months ago

Generally, huge log files like that are full of noise that you don't care about. The last time I had to deal with something like that, I wrote some python scripts to stream in the file and filter the cream from the cruft into a new file.

Eric Richards

SlimDX tutorials - http://www.richardssoftware.net/

Twitter - @EricRichards22

I was about to suggest:

perl -p -i -e 's/search/replace/gi' fileshere

... :)

Advertisement

I know you mentioned explicitly, that the suggested editor should be open source, but I do not know one which handle large files well, but I know of a commercial solution, which I've used successfully for similar tasks as you mentioned...
Been working in telco for a while and had to deal with log files usually bigger than gb trace files, and sometimes had to do complex tasks on them, raging from complex search&replace, pattern matching and filtering tasks. Perl was one solution, that definitely works.

The other one which worked and I loved using it was:
UltraEdit

Works perfectly with gigantic files (it is one of it's advertised selling point), it is a competent source code editor too, and it is script-able. Last time I've used it they had javascript as the scripting engine, but that may have been extended, don't know for sure, but it also worked like a charm...

The tagline may be bit arrogant on their website, but so far, I haven't found a better text editor!
Steep price, but give it a go and try the trial version, it is worth it.


Br.

Blog | Overburdened | KREEP | Memorynth | @blindmessiah777 Magic Item Tech+30% Enhanced GameDev


i mean, some database error, in just few days, could already generate 2 gb or bigger trace files. or log files.

Similar to the people suggesting you pass it through perl, I'd recommend sed (Stream EDitor).

It may take a few minutes to craft a regex that matches your interesting line, but it can quickly filter the multi-gigabyte logfiles into the small number of lines you are interested in.

You discounted Vi or Vim but that is a viable solution to your problem. It may not be the pretty graphical editor you are used to, but it is more graphical than what we had in the early 1980s and it is amazingly powerful.


i mean, some database error, in just few days, could already generate 2 gb or bigger trace files. or log files.

Similar to the people suggesting you pass it through perl, I'd recommend sed (Stream EDitor).

It may take a few minutes to craft a regex that matches your interesting line, but it can quickly filter the multi-gigabyte logfiles into the small number of lines you are interested in.

You discounted Vi or Vim but that is a viable solution to your problem. It may not be the pretty graphical editor you are used to, but it is more graphical than what we had in the early 1980s and it is amazingly powerful.

frob, I know (heard).

The problem here is that I'm not the residential DBA - and I have lots of client. And I have limited hours with each client. Sometimes we did health check, this could be quarterly or less / more frequent depending on the client willing to pay.

since some companies doesn't even have residential DBA - when it come to health check - just drive there, zip all the logs (even a 3 gb could zip into 30 mb due to repetitiveness) and analyze at office. i'll compare between ultraedit and emeditor when i have the time, licensing and pricing and all that.

the thing is that when it come to open source editor, most (which is the pros and cons of open source) focus on the bell and whistles (and maybe more on programming than reading log files) utilities, like opening code bracket, code completion, etc, etc, than supporting large files, regex, etc. most open source editors i've tested choked on large files. atom.io even gives error of 2mb limitation when opening large files.

and i mostly do my work on a windows laptop.

i *may* write my on tools and add functionality as i go foward since i know how to code in Pure Basic (my go to programming tool to write quick little apps / tools). but if there is already open source tools out there, i'm better off using those.

but thanks for all the suggestion everyone.

If you are going to consider commercial solutions, then I'll second UltraEdit. It's an amazing editor, and I've been using it since back in the Windows 3.11 days.

Advertisement

Split it in pieces an editor can handle, then read it.

"I AM ZE EMPRAH OPENGL 3.3 THE CORE, I DEMAND FROM THEE ZE SHADERZ AND MATRIXEZ"

My journals: dustArtemis ECS framework and Making a Terrain Generator

This topic is closed to new replies.

Advertisement