^M Filtering in TeamSite's "Import" function

Has anyone figured out an elegant way to strip the Windows CR/LF characters that appear as ^M's when stored on a Unix Server?

I know you can do a dos2unix, and in FTP you can transfer files in "asci" mode, and in DreamWeaver you can set the "Save As" command to save with '\n' characters, but for users who use Notepad and then do a "File->Import" in the TeamSite GUI, is there anyway to setup that Import utility to convert from Windows CR/LF to Unix '\n' characters?

Thanks in advance,

Tom

Find more posts tagged with

Comments

sajiddc

You can write an event trigger. Basically, whenever a file gets created, it will run the dos2unix script and strip off the ^M characters.

Hope that helps.

Migrateduser

I would caution about doing this as an event trigger. It might work better to do this as part of your submit workflow. for an event trigger you would have to trigger on file creates and modify events (which could get very expensive overhead wise).

With a workflow you could direct this all at once (or maybe introduce a custom menu item that is used in place of import).

The other thing to watch for is that you don't want to run dos2unix on all files. just files that are text (running dos2unix on a binary file or a file that holds binary data could corrupt the file).

There is also the interactive approach of just having a custom cgi menu item that users run on the file when they need to remove the dos formatting. This is probably the one option with the least chance of causing a problem.

mogoo

is dos2unix an out-of-the-box teamsite thing, and if so, where is it located? if not, how do we install it?
in a workflow, would you do a <command> that calls an ipl that calls dos2unix, just like you would a CLT?

thanks,
maureen

akshathp

Which OS are you using? And which version?

Dos2Unix is a unix utility. You would be able to go for the ipl based approach. <command> tag for a call to ipl. This way in the ipl file you could have some validation on the type of file on which you are going to run this command.

Hope this helps.

Akshat Pramod Sharma
Interwoven Inc.

mogoo

Excellent! We're using v5.0 on Solaris. So how do I install the utility, where do I put it?
Thanks,
maureen

akshathp

Before installing, can you please check and confirm if it is already there or not. I am betting it should be already present in /bin or whereever the binaries for Solaris are stored on your server.

Hope that helps.

Akshat Pramod Sharma
Interwoven Inc.

Adam Stoller

It should be in /usr/bin/

As mentioned, however, you want to be sure you only run it on ASCII files - not on binaries - so you could either make it a custom menu item that can be run by the individual who should know what is and isn't binary - you could try to use something like the following (note: only quickly tested):


foreach (@flist){
    print "DIRECTORY: $_\n" and next if (-d $_);
    print "   BINARY: $_\n" and next if (-B $_);
    print "  UNKNOWN: $_\n" and next if (! -f $_);
    print "    ASCII: $_ - running dos2unix...\n";
    `dos2unix $_ > ,x 2>/dev/null`;
    die "dos2unix on '$_' failed" if ($?);
    rename(",x", "$_") || die "[rename] ',x' => '$_' ($?)";
}

or if you want to do it from the command line, without dos2unix - something like:


iwperl -i.bak -n -e 's/\r//; print $_;' filename

--fish
(Interwoven, Curriculum Development)

james1

I would caution you from always copying the dos2unix'ed file over the original file. If there were no ^M's in the file to begin with, then there is no point in overwriting the original file. Especially since the "last modified by" user will now be whatever user that dos2unix ran as.

Here is a simple shell script that I've used for a long time. It overwrites the original file, if and only if dos2unix made changes to the file (i.e., removed ^M characters). Note that it doesn't do the cool file type checking that ghoti's Perl script does. But it should be simple enough to adapt his script to have my script's functionality.

#!/bin/ksh

TEMP_TOKEN=dos2unix.temp

for FILE in $* 
do
        NEW_FILE="$FILE.$TEMP_TOKEN" ;
        dos2unix "$FILE" > "$NEW_FILE" ;
        if diff "$FILE" "$NEW_FILE" > /dev/null
        then
                rm "$NEW_FILE"
        else
                mv "$NEW_FILE" "$FILE"
        fi
done

james1

I forgot to mention that copying the dos2unix'ed file over the original file not only affects the "last modified by", but also the "last modified time"; if the dos2unix is run a while after the file was edited, then this change in "last modified time" may be undesirable, from an audit trail point of view. Same with "last modified by", of course.

-- James