Alternate Data Streams (ADS) allow arbitrary metadata to be associated with files and directories on Windows NTFS. Alternate data streams are the Windows implementation of forks. The apparent size of the file will be unchanged, and most applications and users are unaware of their existence. If a file is moved, any alternate data stream will move along with it, as long as the destination is on an NTFS drive.

The command line can access alternate data streams using redirection operators. Streams are specified on the command line as filename:stream name.

Creating an Alternate Data Stream

As an example, a string is written into an ADS named hidden, which is associated with file test.txt:

C:\test>echo Hidden text > test.txt:hidden

The file appears to be empty, though as detailed below, the metadata is intact and associated with the file:

C:\test>dir test.txt

06/24/2010  01:33 PM                 0 test.txt

Viewing an Alternate Data Stream

The metadata can be viewed by redirecting from it to more:

C:\test>more < test.txt:hidden
Hidden text

The name and content of the ADS can be anything (see 'Details' below for restrictions):

C:\test>echo Arbitrary string > test.txt:arbitraryName

C:\test>more < test.txt:arbitraryName
Arbitrary string

Listing Files With Alternate Data Streams

On Windows Vista and later, a list of alternate data streams can be obtained using DIR /R:

C:\test>dir test.txt /R

06/24/2010  01:33 PM                 0 test.txt
                                    38 test.txt:arbitraryName:$DATA
                                    28 test.txt:hidden:$DATA

On earlier operating systems, the SysInternals utility Streams can be used:

C:\test>c:\tools\SysInternals\streams.exe test.txt

Streams v1.56 - Enumerate alternate NTFS data streams
Copyright (C) 1999-2007 Mark Russinovich
Sysinternals - www.sysinternals.com

C:\test\test.txt:
   :arbitraryName:$DATA 38
          :hidden:$DATA 28

Alternate Data Streams on Directories

Metadata can be added to directories the same way it's added to files:

C:\test>mkdir test2

C:\test>echo ADS on a directory > test2:someText

C:\test>dir /r

06/25/2010  11:27 PM    <DIR>          .
06/25/2010  11:27 PM    <DIR>          ..
06/25/2010  11:27 PM    <DIR>          test2
                                    42 test2:someText:$DATA

C:\test>more < test2:someText
ADS on a directory

Details

Stream Naming

To be more accurate, streams are specified as filename:stream name:stream type. It appears that the only stream type accessible from the command line is $DATA, which is why it's optional. All of the stream types are listed in the WIN32_STREAM_ID structure documentation. The default data stream is unnamed, so filename::$DATA will contain the file's data:

C:\test>echo This is the file > file.txt

C:\test>echo This is the stream > file.txt:stream

C:\test>more < file.txt::$DATA
This is the file

C:\test>more < file.txt:stream:$DATA
This is the stream

Stream names are generally held to the same requirements as any filename. One interesting difference is that stream names can contain characters whose integer representations are in the range from 1 through 31. Refer to Naming Files, Paths, and Namespaces (MSDN) for details.

Note that when using streams with files having a single letter name, the filename should be prefixed with a period and backslash. The reason for this is Windows drive names. For example, does "echo hello > c:test" refer to a stream named test on file c, or does it refer to a file test on drive c?

Executing Streams

As of Windows Vista, it is no longer possible to execute directly from an alternate data stream. On Windows XP and earlier, the Start command was used, similar to start somefile.ext:hiddenExecutable.

Editing with Notepad

Notepad can be used to create and edit alternate data streams. The File Open dialog doesn't recognize stream syntax, however, so the file must be created and opened using command line parameters. Notepad will insist on appending .txt to the stream name.

Programmatic Access

Microsoft provides a sample program in C++, demonstrating how to open and write to an alternate data stream.

Real-World Applications

Downloaded Executables

Since Windows XP SP2, when a file is downloaded from the Internet and executed (assuming a zone-aware browser), this warning is displayed:

Windows displays this warning because the web browser tagged the executable with a alternate data stream named Zone.Identifier:

C:\test>dir /r setup.exe

06/25/2010  12:10 PM           680,467 setup.exe
                                    26 setup.exe:Zone.Identifier:$DATA

By redirecting this stream to more, we can see its contents:

C:\test>more < setup.exe:Zone.Identifier
[ZoneTransfer]
ZoneId=3

The PowerShell blog has more information on zone identifiers.

Viruses

The W2K.Stream virus used alternate data streams.

Additional Resources

• A Programmer's Perspective on NTFS 2000 Part 1: Stream and Hard Link (MSDN, article removed)
• A Programmer's Perspective on NTFS 2000 Part 2: Encryption, Sparseness, and Reparse Points (MSDN, article removed)
• File Streams (MSDN)
• Visual browsing of alternative data-streams in Windows Explorer (CodeProject)
• NTFS Alternate Data Streams (Alex Ionescu)

@echo off

REM ---------------------------------------------------------------------------
REM  Set the following variable to the ffmpeg.exe path on your system.
REM ---------------------------------------------------------------------------
set FFMPEG_PATH="c:\tools\FFmpeg\ffmpeg.exe"

REM make sure the user provided an argument.
if NOT "%~1" == "" goto ARGS_OK

REM bad arguments, print error and exit.
echo.
echo    Usage examples:
echo       mp3toaac.bat file_to_convert.mp3
echo       mp3toaac.bat *.mp3
echo       mp3toaac.bat some_directory\*.mp3
exit /b 1

:ARGS_OK

REM use for/in so that we can accept individual files or wildcards.
for %%i in ("%~1") do %FFMPEG_PATH% -i "%%~i" "%%~ni.aac"

echo.
echo Done.

exit /b 0

Interesting elements of this batch file:

• Use of for/in in order to accept both filenames and wildcards.
• Use of %~X to strip parentheses from a filename variable.
• Use of %~nX to strip the extension from a filename variable (i.e. retrieve the basename).

Terminology Notes

My terminology assumes that you are connecting to a machine on your home network, which is protected by a firewall. However, the diagrams and commands are valid regardless of whether the remote network is at home or not.

Additionally, I use RDCHOST to represent the local name or IP of your RDC server within the home network, and my_home_ip to represent your home IP address as visible from the Internet.

Prerequisites

• Depending on network configuration, port-forwarding may have to be configured on the firewall.
• An SSH client (e.g. Cygwin’s OpenSSH for Windows) must be installed on the local machine.
• An SSH server (e.g. Cygwin’s OpenSSHD for Windows) must be present on some machine within the home network.
• SSH keys must be generated and deployed appropriately.

SSH Command Syntax

ssh -C -N -L localPort:destinationHost:3389 proxyHost

• localPort is the port on localhost through which you wish to connect.
• destinationHost is the Remote Desktop host, as it appears on the home network.
• proxyHost is the host running SSHD, through which you will tunnel.

Configuration 1: Discrete Servers

This configuration has the firewall port-forward SSH to a server on the home network, which proxies the connection to the RDC server:

                       +-----------[ Home Network ]---------------+
                       |                                          |
localhost <----> Home Firewall <---> SSH Server <---> RDC Server  |
                       |                                          |
                       +------------------------------------------+

Command line:

ssh -C -N -L 6009:RDCHOST:3389 my_home_ip

Configuration 2: Combined RDC & SSH Servers

In this configuration, the RDC server also has an SSH server, and the firewall port-forwards directly to it:

                       +------[ Home Network ]-----+
                       |                           |
localhost <----> Home Firewall <-----> RDC & SSH   |
                       |                Server     |
                       +---------------------------+

Command line:

ssh -C -N -L 6009:localhost:3389 my_home_ip

Configuration 3: Firewall as SSH Server

In this configuration, the firewall acts as the SSH server, proxying the connection directly to the RDC server on the home network:

                       +----[ Home Network ]----+
                       |                        |
localhost <----> Home Firewall <---> RDC Server |
                  & SSH Server                  |
                       |                        |
                       +------------------------+

Command line:

ssh -C -N -L 6009:RDCHOST:3389 my_home_ip

Additional Suggestions

Rather than specifying the IP address of your home firewall, I suggest using DynDNS to get a dynamic DNS entry.

2008
   2008_06_27
   2008_06_28
   2008_06_29

First off, I needed to list the files by date:

ls -lt

This showed me that I had files from the 27th, 28th, and 29th in a single directory. So I created a directory for each date.

Next, I used find with the -newer and -exec switches to execute a move command on each file newer than a particular file.

By looking at the ls -lt output, I found the last photo taken on each day. IMG_6369.JPG was the last taken on the 28th, and IMG_6284.JPG was the last taken on the 27th.

To move all of the files taken on the 28th and 29th, I used this command:

find . -type f -newer IMG_6284.JPG -exec mv {} 2008_06_28 \;

Now, all files taken after the 27th are located in the 2008_06_28 directory. Now, I change into that directory, and execute:

find . -type f -newer IMG_6369.JPG -exec mv {} ../2008_06_29 \;

Now I’m left with all the photos from the 27th in the main directory. Those could easily be moved with:

find . -maxdepth 1 -type f -exec mv {} ../2008_06_27 \;

I’m sure there is a quicker, cleaner way to do this, but for the limited number of subdirectories I had to filter into, this was reasonably efficient.

#define VERSION_MAJOR     3
#define VERSION_MINOR     1

By piping this through grep, I can get the line I’m looking for:

c:\>grep "VERSION_MAJOR" version.h
#define VERSION_MAJOR     3

But what I really want is the version number within the line, so I pipe this through sed:

c:\>grep "VERSION_MAJOR" version.h | sed -e "s/^.* VERSION_MAJOR\s*\([0-9]\+\)/\1/g"
3

So now we have extracted the version number corresponding to “VERSION_MAJOR”. However, in order to conditionally branch in our batch script, we need to get this value into an environmental variable. In bash, this would be accomplished with backticks. As it turns out, Windows batch files do provide similar functionality, but through a much clumsier means:

c:\>for /f "tokens=3" %i in ('grep " VERSION_MAJOR" version.h') do set MAJORVERSION=%i
c:\>echo %MAJORVERSION%
3

• \f – form feed
• \n – new line
• \r – return

Since tr is deleting characters, not strings, we can simply specify all of these in a single command:

cat input.txt | tr -d \r\n\f > output.txt

Another way to assemble this command:

tr -d \r\n\f < input.txt > output.txt