M-x all-things-emacs

Quick Tip: dos2unix, et al

April 30th, 2007 by Ryan McGeary · 47 Comments

I despise the fact that we live in a world with different end-of-line file formats. Windows/DOS uses CRLF, Unix uses LF, and Mac’s used to use CR1. Thankfully, Mac’s started to adopt the Unix format when OS X was released — if only Windows could do the same.

What I despise even more is that some editors seem to be incapable of determining the difference between a DOS and Unix file. There’s nothing worse than finding a once, perfect Unix file corrupted by a small section of lines with CRLFs while the rest of the file keeps only LFs. Most of the time, the blame can be placed on one’s editor configuration, but I also blame some editor defaults for not at least maintaining the format that the file was opened in. To be fair, most power-editors like emacs, vim, TextMate, etc behave “correctly” by default and keep the format that the file was opened in, but many others (unnamed) do not.

There’s not a whole lot we can do to avoid these problems without hounding our peers, but there are ways to fix these problems after they’re found.

Let’s fix the nastier problem first. When you find a file corrupted with half LFs and half CRLFs, strip out the ^M (CR) characters with a quick search and replace. Run M-% (query-replace) and substitute C-q C-m with nothing. C-q runs quoted-insert and is useful for inserting control characters (e.g. ^M, entered as C-m). Afterwards hit the exclamation point (!) to tell query-replace to replace all matches with no questions.

Other times, you will run into DOS formatted files and will just want to convert them to Unix format for consistency sake. To do this, open the buffer and run C-x <RET> f then enter unix or undecided-unix when prompted for the new coding system. This runs set-buffer-file-coding-system and the result is very similar to running dos2unix myfile.txt at the command line.

1 CR is Carriage Return. LF is Line Feed (aka Newline).

Tags: osx · quick · tips · unix · windows

47 responses so far ↓

  • 1 James // Apr 30, 2007 at 3:27 pm

    Fantastic. I never knew that ‘!’ turns a q-r-r into a replace-regexp. I’ve always just done a C-g, then run a replace-regexp reusing the last replacement.

  • 2 Peter // Apr 30, 2007 at 3:50 pm

    Have you seen the package: http://centaur.maths.qmul.ac.uk/Emacs/files/eol-conversion.el

  • 3 Christoph // May 1, 2007 at 9:03 pm

    C-x <RET> C-f in fact is C-x <RET> f.

  • 4 Ryan McGeary // May 2, 2007 at 12:39 am

    Thanks Christoph. I fixed the typo. I’m sorry if this caused anyone else unnecessary confusion.

  • 5 David Rysdam // Jul 22, 2010 at 3:30 pm

    Just found your blog and it’s great. Only problem: How am I going to remember these tips when I need them?

    Anyway: Run M-% (query-replace) and substitute C-q C-m with nothing. ... Afterwards hit the exclamation point (!) to tell query-replace to replace all matches with no questions.

    Why not M-x replace-string and then no (!)?

  • 6 David Rysdam // Jul 22, 2010 at 3:32 pm

    Oh man, I thought this entry was from a few months ago but it’s actually 3 years old. Errr…

  • 7 Ryan McGeary // Jul 23, 2010 at 12:28 am

    David,

    Good point, but mostly, it’s because I don’t have replace-string bound to a key binding. M-% is ingrained in my typical workflow.

  • 8 anon // Jun 28, 2012 at 12:56 am

    Tips can easily be converted to a macro and saved; thus reused

  • 9 Personal Trainer Seattle // Jul 22, 2015 at 12:44 am

    ... [Trackback]

    [...] There you will find 27778 more Infos: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 10 Con-Cret Pre Workout // Jul 23, 2015 at 4:24 pm

    ... [Trackback]

    [...] Read More here: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 11 amazin gas station video // Jul 24, 2015 at 9:00 am

    ... [Trackback]

    [...] Read More Infos here: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 12 amazin gas station video // Jul 24, 2015 at 10:35 am

    ... [Trackback]

    [...] Read More here: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 13 criminal defense law firm // Jul 24, 2015 at 9:33 pm

    ... [Trackback]

    [...] Informations on that Topic: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 14 criminal attorney // Jul 24, 2015 at 11:33 pm

    ... [Trackback]

    [...] Informations on that Topic: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 15 Atlanta car service // Jul 25, 2015 at 1:43 pm

    ... [Trackback]

    [...] Informations on that Topic: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 16 Locksmith Orangevale // Jul 26, 2015 at 6:19 am

    ... [Trackback]

    [...] There you will find 39595 more Infos: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 17 Roseville Locksmith // Jul 26, 2015 at 6:19 am

    ... [Trackback]

    [...] There you will find 78016 more Infos: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 18 buy propecia 5mg australia // Jul 27, 2015 at 9:03 am

    ... [Trackback]

    [...] There you will find 67034 more Infos: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 19 cleaning agency London // Jul 27, 2015 at 6:21 pm

    ... [Trackback]

    [...] There you will find 68799 more Infos: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 20 online surf store // Jul 28, 2015 at 7:02 am

    ... [Trackback]

    [...] There you will find 35232 more Infos: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 21 Insider Binary Bot // Jul 31, 2015 at 3:43 pm

    ... [Trackback]

    [...] Read More here: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 22 family chiropractor Omaha // Jul 31, 2015 at 6:24 pm

    ... [Trackback]

    [...] Informations on that Topic: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 23 Prodaja staklene ambalaze // Aug 1, 2015 at 4:34 am

    ... [Trackback]

    [...] There you will find 92097 more Infos: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 24 it security // Aug 1, 2015 at 8:21 pm

    ... [Trackback]

    [...] Informations on that Topic: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 25 http://tiptopplumbers.com // Aug 2, 2015 at 6:14 pm

    ... [Trackback]

    [...] Informations on that Topic: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 26 http://www.certifiedhandymen.net // Aug 2, 2015 at 7:30 pm

    ... [Trackback]

    [...] Informations on that Topic: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 27 http://www.optimumcontractors.com // Aug 2, 2015 at 8:20 pm

    ... [Trackback]

    [...] Read More here: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 28 social signals Twitter // Aug 2, 2015 at 10:24 pm

    ... [Trackback]

    [...] Read More: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 29 imp source // Aug 2, 2015 at 11:29 pm

    ... [Trackback]

    [...] Read More Infos here: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 30 http://inhalefreely.com/ // Aug 3, 2015 at 1:42 pm

    ... [Trackback]

    [...] Read More Infos here: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 31 electronic water heater repair // Aug 4, 2015 at 1:08 am

    ... [Trackback]

    [...] Find More Informations here: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 32 What is the advance measurement system? // Aug 4, 2015 at 1:23 am

    ... [Trackback]

    [...] Read More: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 33 i thought about this // Aug 4, 2015 at 2:19 am

    ... [Trackback]

    [...] Read More here: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 34 www.supremecontracting.net // Aug 4, 2015 at 3:08 am

    ... [Trackback]

    [...] There you will find 36002 more Infos: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 35 tankless hot water // Aug 5, 2015 at 7:10 am

    ... [Trackback]

    [...] Read More: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 36 Will I be able to sell my home after the repairs are done? // Aug 5, 2015 at 10:14 am

    ... [Trackback]

    [...] Read More here: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 37 mannequins // Aug 6, 2015 at 8:48 pm

    ... [Trackback]

    [...] Find More Informations here: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 38 click this link here now // Aug 9, 2015 at 6:23 pm

    ... [Trackback]

    [...] Read More Infos here: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 39 their explanation // Aug 9, 2015 at 6:49 pm

    ... [Trackback]

    [...] Informations on that Topic: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 40 ttt4 // Aug 10, 2015 at 10:17 pm

    ... [Trackback]

    [...] Read More here: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 41 mahjong gratuit // Aug 11, 2015 at 12:04 am

    ... [Trackback]

    [...] Read More here: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 42 orthodontiste Lille // Aug 12, 2015 at 6:14 pm

    ... [Trackback]

    [...] Read More Infos here: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 43 find here // Aug 12, 2015 at 10:15 pm

    ... [Trackback]

    [...] Informations on that Topic: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 44 trending topics // Aug 13, 2015 at 12:22 am

    ... [Trackback]

    [...] Read More Infos here: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 45 electrical upgrades // Aug 13, 2015 at 11:50 pm

    ... [Trackback]

    [...] Read More Infos here: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 46 Tree Root Removal and Prevention // Aug 13, 2015 at 11:54 pm

    ... [Trackback]

    [...] Find More Informations here: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]

  • 47 dental implants // Aug 14, 2015 at 12:17 am

    ... [Trackback]

    [...] Read More Infos here: emacsblog.org/2007/04/30/quick-tip-dos2unix-et-al/ [...]