DATUP Version 3.13 by Scott Botterud

I       OVERVIEW: What is DATUP?
II      INSTALLATION: Step by step installation, configuration, and testing instructions.
III     Revision History: What's new.

DATUP is a utility designed to automate the process of downloading the latest virus definition (DAT) files from the McAfee FTP server and apply them. The necessity to keep the latest virus definition files in place is paramount, particularly in business environments. Having an older DAT can render your antivirus engine ineffective against the latest viruses, and can mean the difference between keeping your system clean and getting infected. DATUP can run on Windows based Workstations or Servers (9x, NT, ME, 2000, XP).

DATUP.BAT is a sophisticated batch file. It first sets the system clock, then logs into the McAfee FTP server and checks if a newer version of the .DAT file are available. If a newer file is available, it is downloaded, unachieved, and copied to the appropriate location(s). DATUP then cleans up after itself and finally sends an email message reporting all activity. DATUP is automated, typically running every day at a given time(s). We use this utility to update our email server, the public DAT pool, and our Novell Netware File server running the McAfee NETSHIELD program.

This program was originally developed for use with VPOPSCAN, another freeware program available from http://www.info-techs.com . This archive includes an additional file, DATUPNOV.ZIP which contains the additional code for automating the update of .DAT files on a Novell Netware file server running the server based McAfee NetShield antivirus program (an excellent Server based antivirus program). The latest versions of out software can be downloaded from: http://www.info-techs.com/download.shtml .

This guide is designed to walk you through the installation and testing necessary to get DATUP working successfully.
There are three steps to installing DATUP:
1)  Modify the DATUP.BAT defaults.
2)  Copy files to the correct locations.
3)  Set up WinCron to automate the DATUP process.
4)  Add NETSHIELD for Netware support (optional)
DATUP.BAT is heavily commented and should be pretty self explanatory. The following lines of DATUP.BAT will need to be modified:
@ECHO OFF								
REM Set the home directory where DATUP lives				
SET HOME=C:\UTIL\DATUP					<-- Set this to the location of DATUP.BAT
REM EMail Recipient to send all message
SET EMR=scott						<-- Send email to this person
REM EMail Server to use
SET EMS=					<-- Email Server to use
SET TZ=PST8PDT						<-- The Time Zone for UNZUP to use
Details for each field:

This is where DATUP.BAT lives. I suggest placing DATUP.BAT in it's own separated directory. Note that DATUP.BAT will create an UPDATE directory and a few files for it's own use. Do NOT include a trailing backslash ("\").

SET EMR=scott
The EMail Recipient that is to receive email messages from DATUP. Depending on what email server used, you may need to include the @someserver.com suffix to the name.

The EMail Server POSTIE should use to send messages. This is your SMTP server's IP address or the full name. If your SMTP server requires authentication you will need to add the "-pass:password" field to the POSTIE.EXE arguments.

The Time Zone for UNZIP.EXE to use. This field is REQUIRED! If it is removed UNZIP.EXE will exit with an ERRORLEVEL=1 even on successful a successful unarchive. Do not change unless you have a good reason.

Copying the necessary files to the proper locations is important for the proper operation of DATUP. This distribution includes all the necessary files, already unachieved, along with the original archive files (per the Shareware, Freeware or GNU licenses).

1) WGET.EXE - This is the FTP client. DATUP.ZIP includes the distribution copy of WGET.ZIP (WGET-1.8.1B.ZIP), as well as the single unachieved executable program file WGET.EXE.
Copy WGET.EXE file to one of the following locations:
A) The DATUP home directory (i.e. C:\UTIL\DATUP).
B) Somewhere in the PATH. WGET.EXE can be called from any PATH location. This location can be a File Server, or on the Workstations hard drive. If you are uncertain what the PATH for your Windows machine is, open a DOS windows (Click on START, then RUN, and type COMMAND) and type PATH. We recommend copying WGET.EXE to a location that gets backed up.

2) POSTIE.EXE - This is the program DATUP uses for sending email messages. The DATUP*.ZIP distribution includes the POSTIE.ZIP file, which contains the distribution copy of POSTIE.EXE, as well as the executable program file, POSTIE.EXE. We have done this because POSTIE is a self extracting and self installing program. Some may not want to go through the installation process. The POSTIE.EXE distribution file also has the quirk that the self extracting file, POSTIE.EXE is also the name of the main program file, POSTIE.EXE, so when the self extraction and installation process runs, it reports that the POSTIE.EXE already exists and do you want to overwrite.....

Copy POSTIE.EXE - Copy this file to either the DATUP home directory or somewhere in the PATH. Again we recommend copying POSTIE.EXE to a location that gets backed up. POSTIE is configured from the DATUP.BAT file and does not require any .INI files.

3) Copy POSTIE.TXT - The POSTIE documentation. Although not necessary we recommend copying this file to the same location as the POSTIE.EXE program file so it is easy to find.

4) Copy TDEL.EXE - This is the utility DATUP uses to delete files. Copy this file to either the DATUP home directory or somewhere in the PATH.

5) Copy CMDTIME3.EXE - This is the utility DATUP uses to synchronize the system time. Copy this file to either the DATUP home directory or somewhere in the PATH.

6) Copy UPTIME.EXE - This is the utility DATUP used to find how long the system has been running since the last boot. Copy this file to either the DATUP home directory or somewhere in the PATH.

7) COPY the rest of the files: Copy the following to the DATUP home directory:
	DATUP.BAT		<-- The BATCH file that runs the whole mess (REQUIRED)
	DATVER.ENV		<-- Used to pass environment variables to Windows (REQUIRED)
	README.TXT		<-- The file you are reading.
The above files are enough to get DATUP working. Verify that DATUP is indeed working for you as you wish. After it's proper operation is verified, you can automate the DATUP process so no human intervention is required for proper DATUP operation. To do that goto STEP 3:

STEP 3: SET UP WINCRON: WINCRON is used to automate the DATUP process. It is a port of the UNIX CROM utility for Windows. There are a few steps to installing this utility:

A) Unachieved WINCRON.ZIP to a temporary directory. This will contain six files:
	WINCRON.EXE	<-- The main program file. 
	CRON		<-- A sample configuration file for WinCRON
	WINCRON.LNK	<-- A sample shortcut to start WinCRON 
	WINCRON.HTML	<-- Instruction for WinCRON in HTML
	MSVBVM60.DLL 	<-- An updated Visual Basic .DLL library file
	OLEAUT32.DLL 	<-- Another updated Visual Basic .DLL library file
B) Copy WINCRON.EXE - This file can go either in the DATUP Home directory or somewhere in the path.

C) MSVBVM60.DLL and OLEAUT32.DLL - These are updated versions for the Visual Basic .DLL library. You may or may not need these files. To test if you need the updates, simply run WINCRON.EXE. If it reports the libraries are out of date you will need to apply these updates. They will work for Windows 9x or NT. The updating of these files is a little involved, as Windows uses one or both of then when it starts up, so Windows keeps these files open. This means usually you cannot simply copy or overwrite them, as you will get a sharing violation. The process I have had to use is to copy the files to a temporary directory (lets say C:\1) and then reboot Windows. After the POST test, just after the BEEP, hold down the F8 key. This will give you a BOOT menu. Select STEP-BY-STEP and hold down the N key to get to a DOS prompt. Copy the two files from C:\1 to C:\WINDOWS\SYSTEM and reboot your computer. If you are running NT you will have to boot from a system disk and follow the same copy procedure. If you do not have a Windows BOOT disk you can download the image files and the utility to create a Windows 98SE disk from http://www.info-techs.com/download.shtml . If there is a better way to do this I would love to hear it: scott@info-techs.com

D) CRON - Copy this file to the DATUP home directory. This is the configuration file for WinCRON. It tells WinCRON when it should run and what to do. The WINCRON.HTML file explains in depth the configuration of this file.
The CRON file is set up to run WinCRON at 4:00 AM every day:
To run DATUP every 4 hours (6 times a day) every day of the week:
		00 00,04,08,12,16,20 * * 0,1,2,3,4,5,6 
Note that Windows NT and 9x require different configurations. Use any text editor to modify CRON as needed.

E) WINCRON.LNK - This file is typically copied to the STARTUP folder:
This allows Windows to load this file automatically at startup. WINCRON.LNK has come command line arguments that tell WinCRON how to load:
The following explain the command line:
	C:\UTIL\BACKUP\CRON	<-- The location of CRON (The WinCRON configuration file).
	/Log			<-- Log activity to a file - Good for keeping track of the program.
	/NoUpdate		<-- WinCRON will not keep pooling CRON for updates or changes.
	/Hide			<-- Loads as a background process, so there is no icon. Run the Task Manager to view. 
For more information on the command line arguments please see WINCRON.HTML. To modify WINCRON.LNK, Right Click on it and select PROPERTIES. The Icon is in WINCRON.EXE.

F) WINCRON.HTML - The documentation file for WinCRON. Not necessary, but we recommend copying this file to the same location as WINCRON.EXE so it is easy to find.

G) Verify WINCRON is configured and operating properly:

The final step in setting up DATUP is to automate the entire process. Although Windows does have a built in task scheduler, we found it rather clunkie, unreliable, and difficult to configure (especially from NT). The freeware program WINCRON is an easy, effective, and reliable solution that consumes a small amount of resources. Testing WinCron takes a little patience. First off, do NOT use the WINCRON.LNK file to test WinCron, as WINCRON.LNK loads WinCRON as a background process, so you will not be able to see what is going on. We recommend running the WINCRON.EXE from a test batch file for testing purposes. Use something like the following for TESTCRON.BAT:
	C:\UTIL\BACKUP\CRON	<-- The location of the CRON file.
Modify the CRON file so the it will run 1 or 2 minutes in the future. Run the TESTCRON.BAT file and verify that it is behaving as expected. Please note that you may have to use the Task Manager (CTRL-ALT-DEL) to terminate WinCRON between each test, and reload it again so it will see changes to the CRON file. Once you are satisfied that WinCRON is functioning properly, you can then allow it to run from the WINCRON.LNK file in the STARTUP folder. With the command line arguments, it will load as a background process and will not show a desktop icon or any proof that it is loaded other than looking at the Task Manager.

STEP 4) NOVELL NETSHIELD: If you are running the McAfee NETSHIELD program on a Novell Netware File Server, the DAT files can be updated directly from DATUP.BAT. Please refer to the DATUPNOV.ZIP file for complete instructions on adding this feature.

I hope you find this utility useful and a time saver. If you have any revisions, suggestions, or questions please feel free to contact me:


Version 3.13 (Released 1/28/2004)
This is a bug fix of version 3.12:

1) The McAfee FTP Server (ftp.mcafee.com) has changed and no longer accepts passive FTP transfers. For those users needing passive FTP, (such as those behind a NAT) we suggest using the Network Associates FTP server (ftp.nai.com). Network Associates owns McAfee and their server is a mirror of McAfee's. The following changes were made to DATUP.BAT:
	WGET.EXE --passive-ftp -nc ftp://ftp.mcafee.com/pub/datfiles/english/dat-*.zip
was changed to:
	ECHO ftp.mcafee.com does not like PASSIVE FTP transfers, while nai.com does. NAT users may find PASSIVE works better.
	WGET.EXE --passive-ftp -nc ftp://ftp.nai.com/pub/datfiles/english/dat-*.zip
	REM WGET.EXE -nc ftp://ftp.mcafee.com/pub/datfiles/english/dat-*.zip
This allows users to choose which FTP server and mode to download the latest DAT files.

2) Changed the command line for CMDTIME3.EXE. It was found that CMDTIME3.EXE would occasionally hang with the old command line spelling out the time server. The /Q switch is for quick mode where CMDTIME3.EXE pools time.nist.gov (default). The changes are:
	CMDTIME3.EXE time.nist.gov /M:65 SYNC>>timesync-log.txt
	CMDTIME3.EXE /Q /M:65 SYNC>>timesync-log.txt

Version 3.12 (Released 11/6/2003)
This is a feature enhancement of version 3.11:

1) Changed the method of testing for a bad ZIP file from simply listing the contents of the ZIP file to actually performing an integrity test of the ZIP. The code changed from:
	UNZIP.EXE -l %HOME%\DAT*.ZIP>unzip-log.txt
	UNZIP.EXE -t %HOME%\DAT*.ZIP>unzip-log.txt

Version 3.11 (Released 9/3/2003)
This is a bug fix for version 3.1:

1) The original code to create the NEW.TXT file for a fresh installation was:
	IF NOT EXIST %HOME%\new.txt DIR %HOME%\dat*.zip /b >%HOME%\new.txt 
This worked fine under NT but kacked under Windows 9x. The new code is:
	DIR %HOME%\dat*.zip /b > %HOME%\new.txt
This has proven to work on all tested platforms.

2) The DATUP.BAT file in the 3.1 distro was missing the updated code for TDEL.EXE.

3) Eliminated a left over ECHO ERRORLEVEL=%ERRORLEVEL% (BTW, this only works under NT, 2K & XP)

4) Switched from text documentation to HTML documentation. Hope this is easier to read, print, search and navigate!

Version 3.1 (Released 9/2/2/2003):
The feature bloat continues:

1) Added the CMDTIME Utility: CMDTIME3.EXE (http://www.softshape.com/download/) is used to set the system time to an internet atomic clock. This was added to keep the email server time accurate, therefor keeping the time stamp on messages correct. For most users in the US, time.nist.gov is a good choice for the time server. The output of CMDTIME3.EXE is now part of the email DATUP generates, letting the recipient know how much time drift the system had.

2) Added the UPTIME Utility: UPTIME.EXE (http://thunder.prohosting.com/~ladi/e_cmd32.html#top) has been added to the mix. This allows the recipient of all the email messages DATUP generates to monitor the uptime of the system DATUP is running on. If there is an unexpected reboot, this could indicate a possible security problem, defective UPS, or tampering. Many of the new viruses need a reboot to activate.

Version 3.0 (Released 8/26/2003):
Post the SoBig and Blaster virus attacks. A number of the programs that DATUP uses have been changed, new features have been added, exhaustive error checking and handling, along with a much easier installation and configuration process:

1) Change the FTP client program: The McAfee (ftp.mcafee.com) and Network Associates Inc (ftp.nai.com) FTP servers have suddenly become problematic for NCFTPGET.EXE to talk to. NCFTPGET keeps timing out when attempting to negotiate a connection. After some digging I found that McAfee and NAI are now (?) running Windows 2000 FTP servers (ug). After perusing some user groups I found others were having similar problems with NCFTPGET.EXE, so it was time to move away from NCFTPGET.EXE.

I have opted to go with the GNU program WGET.EXE (http://gnu.mirror.widexs.nl/software/wget/wget.html). This works fine with both Windows and UNIX/LINUX based servers, does not require an installation, and includes the source code. The performance, options, and progress indicators of the FTP session is far superior to the old NCFTPGET.EXE, and it works 100% of the time.

One thing noteworthy is WGET uses a very small amount of bandwidth to check for updates. Because of the speed the new generation of viruses are moving at, it is imperative to get the latest .DAT file downloaded and in place as soon as possible to help protect your system. Originally we suggested scheduling DATUP to run once a day. Because of the low demands of DATUP we now suggest multiple DATUP sessions be performed daily. See the WinCRON section for assistance.

2) Put in error checking for WGET.EXE: In case the FTP server is too busy, down, or inaccessible, WGET.EXE will retry the transfer 20 times (WGET default setting) after which WGET.EXE is then looped another 10 times, for a total of 200 iterations. If WGET still cannot get through it will send an email to the system administrator (?) notifying them of the FTP problem along with a debug log file from WGET.EXE. On exit the system's PC speaker will signal an alarm by beeping 8 times.

3) Put in ZIP file integrity checking: The shareware PKUNZIP.EXE has been replaced with the open source UNZIP version 5.5 by Info-ZIP (http://www.info-zip.org/). This was done because we wanted to error check the downloaded DAT*.ZIP file. On occasion a bad DAT*.ZIP file has been downloaded. The old PKUNZIP.EXE would attempt to unarchive the bad ZIP and then prompt for user input (Errors were found in .ZIP file, attempt to fix (<Y>es/<N>o)?). There was no way to get around this shortcoming of PKUNZIP to require human intervention, which would stopped the whole mess in it's tracks.

The new UNZIP.EXE program gracefully handles such situations by simply exiting with a message and an errorlevel>0. The new UNZIP.EXE unarchiver is also a lot faster, has a smaller memory footprint, and is GNU open source. If a bad DAT*.ZIP file is found the system will delete the defective archive, wait 5 minutes, and retry the download and unarchive process a total of 10 times. If after the 10 iterations it still was unsuccessful, an email is sent to the admin (?) with the ZIP log attached. On exit the system's PC speaker will signal an alarm by beeping 8 times.

4) Update the old WAIT!.EXE program from Mustang Software to the freeware WAIT.EXE program. This new WAIT.EXE has the advantages of a pleasant countdown display and the ability to cancel the wait process by pressing any key.

5) Renamed FC.EXE to FILECOMP.EXE. Windows NT already has an FC.EXE, and this caused some confusion, besides which the m$ version will not work as I need. FILECOMP.EXE will work fine from anywhere in the path.

6) Improved code: Replaced the hard coded data for the following parameters with environment variables:
	%HOME%	= DATUP home directory (typically C:\UTIL\DATUP) 
	%EMR%	= Email Recipient for all messages
	%EMS%	= Email Server to send all messages.
Putting these parameters into environment variables makes it a lot easier and faster to get DATUP working under a variety of configurations. These variables are declared at the beginning of the batch file.

7) Improved Compatibility: The use of the built in Windows programs DELTREE and DEL has been replaced with the excellent freeware program TDEL.EXE (http://tutils.cjb.net/tdel.htm). The reason being, on Windows 9x and ME the operation of DELTREE and DEL worked differently than Windows NT/2000/XP. This made for troublesome installations, necessitating changing the DATUP code to make it work on different systems. Regardless of what Operating system DATUP is run on, it should now be able to now properly operate without any code modifications. TDEL.EXE is small, has a plethora of options, and can work properly from the path.

8) Automatic Directory Creation: DATUP now creates the UPDATE directory on it's own if it does not exist. Duh..

9) If the DATVER.ENV file is found to be missing the system will now exit and send an email notification to the EMR that the file needs to be fixed.
10) Improved email notification: A variety of additional information is now emailed to the EMR (EMail Recipient):
A) Email notification of new DAT files now lists the old and new ZIP file names.
B) Email notification of new DAT files additionally lists the system UpTime, and the results of TimeSync.
C) If the DATVER.ENV file is missing the system will now exit and send the EMR an email informing them of the missing file.
D) If WGET encounters any problems downloading the DAT file, it goes into DEBUG mode. In DEBUG mode all screen output is redirected to file (wget-log.txt), which is sent to the EMR.
E) If UNZUP encounters a problem it's output file (unzip-log.txt) is sent to the EMR.
11) Improved screen output: DATUP.BAT now runs with ECHO OFF. The system will report it's progress on screen in easily understood terms. This is much better for monitoring or troubleshooting.

12) Audio Notification of Problems: If DATUP encounters a serious problem it will beep the PC speaker 8 times to hopefully inform someone that something is wrong.

13) DATUP Code for Novell: The code for the NOVELL portion of DATUP is now included in DATUP.BAT. It is by default bypassed, but can easily be enabled for easier implementation of the Novell functions.

14) Updated DATUPNOV.ZIP file: The DATUPNOV.ZIP file now contains the distribution file NTS.ZIP (Novell Time Sync) so it does not have to be downloaded.

15) Novell Netshield timing fixes: Some possible timing issues were addressed for the Novell side of DATUP. The WAIT periods have bee increased to allow the server to complete it's load and unload process under high demand circumstances.

Version 2.2 (Released 4/2/2002):
After applying version 2.0 to a number of servers we found a few problems. Version 2.2 addresses the following problems:

1) We fixed the command line for calling POSTIE.EXE:
		C:\UTIL\BACKUP\POSTIE.EXE -host: .........
		C:\UTIL\DATUP\POSTIE.EXE -host: .........
2) When running POSTIE.EXE, if you get an error message that WS2_32.DLL is missing, you have an older version of Windows 95, and need to apply the Windows Socket 2 Update. This can be downloaded from: http://www.microsoft.com/windows95/downloads/contents/WUAdminTools/S_WUNetworkingTools/W95Sockets2/Default.asp It is easy to install, creates a backup of all files, but requires rebooting.

3) The initial DATUP.BAT file used COMP.EXE to compare files. We found this unreliable under Windows 95. It has been replaced with FC.EXe (File Compare). DATUP.BAT was updated:

		ECHO N|COMP c:\util\datup\old.txt c:\util\datup\new.txt

		FC c:\util\datup\old.txt c:\util\datup\new.txt
This works well under all tested operating systems and does not require the "N" pipe. We have included a copy of FC.EXE that does not seem to care what Operating System it operates under.

4) Updated the command to check if this is a fresh installation, as it was problematic under some versions of Windows. DATUP.BAT was updated:
We found some versions of Windows just did not work correctly with the DIR command to see if the DAT-*.ZIP file was present.

Version 2.1 (Released 3/21/2002):
We have added the ability for DATUP to update the virus definition files on Novell Netware 3.x File Server running the McAfee NETSHIELD program. We have left this as a separate file (DATUPNOV.ZIP) that can easily be added to the DATUP.BAT file if you have a Netware Server.

Version 2.0 (released 3/18/2002):
This is the first non Beta release, and addresses a number of inefficient and weak aspects of the initial Version 1.0, along with adding features:

1) NEW FTP CLIENT: DATUP now uses a new, freeware FTP program called NCFTP. The original version 1.0 of DATUP used the built in Windows FTP client program. The new FTP program, NCFTP, has a number of features that make is a lot better choice over the Windows FTP client.

A) When called, NCFTP logs in to the FTP server and checks if the version of the file to be downloaded is the same or newer. If there is a new DAT file available, NCFTP will download it, and if not, it will log off. This was one of the weaknesses of version 1.0, where it would go out and blindly download the dat-*.ZIP file, regardless if it was newer or not. This was a waste of bandwidth. Because this release takes very little bandwidth, I don't feel bad about allowing it to run more than once a day. With the speed that viruses spread and propagate these days it may well be necessary B) NCFTP also reports it's progress as to how big the file is, remaining time, download speed, etc. A nice feature. The NCFTP program is freeware. Many thanks to the author, Mike Gleason, for the creation of his excellent Freeware program. Because the program is a self extracting and self installing .EXE file, some may not want to go through the installation process, so we have included the NCFTPGET.EXE file already extracted. The latest version can be downloaded at http://www.ncftp.com/ncftp/

2) Checks if this is a new install. The old version would fumble the first couple of attempts.

3) Performs version checks to see if it is necessary to unzip and copy all the files. The old version would overwrite everything and copy everything.

4) The program now deletes the old dat-*.zip files, cleaning up after itself.

5) Added the function of sending an email notifying me that a new version of the DAT files was downloaded and applied, and reporting the current version of the DAT files (ie DAT-4191.ZIP). This assures me that indeed the program is doing it's job of keeping the virus definition fines up to date. Many thanks to the author of Postie, Andrew Davison, for the creation and release of his excellent Freeware program. Get the latest version at: http://www.infradig.com/infradig/postie/index.shtml

Version 1.0 (released 2/28/2002):
The first version. It was heavy handed and crude but it worked. The original code was:
FTP -s:dat.txt -a ftp.mcafee.com

cd pub/datfiles/english
mget dat-4*.zip