Market Analysis System (MAS): Frequently Asked (or Anticipated) Questions

Contents

[Home]
Operating the Server (General)
Operating the Server (Windows)
Administration and Setup
Data-related Problems
The MAS Charting GUI
Database Problems
Technical Analysis Indicators
Signal Generation and Market Analyzers

Operating the Server (General)

Once the server is started in the background, how do I stop it.

You can stop the server by starting up the command-line client (either macl - or maclj, if you don't have Python installed but do have Java) and then entering a Control-E and a newline.

When I change the data files after the server is run, it will not pick up any newly named files. (For example, if ibm.txt and qqq.txt were listed when I ran the MAS server, and I added nt.txt later, MAS will not pick up 'nt'.) Is there any way without resetting (shutdown and restart) the server for it to pick up the new data files?

Currently, the only way to add new files is to shut down and restart the server. The capability to add new tradables while the server is running is a feature that may be added in the future if there is enough demand for it.

What are the -d and -i options for?

The -d and -i options are intended to be used together. When the data is being read from files, it allows the server to figure out whether a file holds daily or intraday data. For example, if daily data lives in files with the pattern *.daily and intraday data in files with the pattern *.5-minute (for 5-minute data in this example), you would include the options "-d daily -i 5-minute" when you start the server.

Operating the Server (Windows)

I`ve tried to install MAS 1.6.5 for Windows but am not able to get the charting application working. I installed Java, as instructed in the readme file, but when I click on the "MAS Charts" icon I get this error message:

c:\MAS\mas1.6.5\lib\classes>java MA_Client localhorst 18273
java.io.IOException: Couldn't get I/O for the connection to: localhorst

This error message indicates that the MAS charting application cannot connect with the MAS server. Most likely this is because you have not started the server.

In order to start the charting application you first need to start the component called the MAS server. (This is the component of MAS that does most of the calculations.) This is done by double-clicking on the "MAS Server (web)" icon (if you wish to get data from the web) or the "MAS Server (files)" icon (if your data resides on files on your computer) to start the server. After the server has started, double-click on the "MAS Charts" icon to start the charting application.

Administration and Setup

I installed MAS and when I try to run it, I get the following error [or something similar]:

mas: error while loading shared libraries: libodbc.so.1: cannot open shared object file: No such file or directory

The cause of this error is that the shared library file libodbc.so.1 is not where MAS expects it to be. (Usually it's in /usr/lib.) Unless you're using an old Linux distribution, there's a very good chance that this file is included in the distribution, but you probably did not install the package that includes the file. So the first thing to do is to start up your distribution's package install program, look for the package that contains the libodbc.so.1 file, and install it. (Note: Since libodbc is an ODBC library used with database systems, you're likely to find it in a database-related package. For example, one Redhat 8.0 user apparently found it in the "databases" package.)

If you can't find the file in your distribution, you can try searching for it on the web. google.com may work for this purpose, but a better solution is probably rpmfind (and search for "libodbc.so" or "libodbc.so.1"). Chances are good that you will be able to find a libodbc.so.1 that works with your system, but if not, one last thing to try is to use the libodbc.so.1 file that comes with the MAS installation. Once you find that file (by default, the rpm installation puts it in /opt/mas/lib), you need to log in as root and do the following:

ln -s /opt/mas/lib/libodbc.so.1 /usr/lib

(Substitute /opt/mas/lib/libodbc.so.1 with the path for this file on your system.) If MAS still gives an error about libodbc.so.1, then the version of the file that came with MAS is probably not compatible with your system. The only fix is to find a version that is compatible with your system and install it.

I'm running into problems with MAS not using the new mailer I've set up - it is still trying to use elm. I have set the MAS_MAILER variable: MAS_MAILER=/home/masuser/masconfig/masfilter. masfilter is executable. When I do the market analysis study, I get sh: elm not found (since I do not have elm installed on my machine). What am I doing wrong?

Most likely you did not export the MAS_MAILER environment variable. For example, if you are setting the variable in your .profile (or whatever login initialization file you're using with your bash/sh-compatible shell), you would add the following line to your .profile file:

export MAS_MAILER=/mailerpath

or, for csh-compatible shells:

setenv MAS_MAILER  /mailerpath

Of course, you also need to export the MAS_DIRECTORY variable if you are setting it.

If this is not the problem - if you've already exported the MAS_MAILER variable, then the problem could be that you are using a user (event registrant) for your events that was set up with a previous MAS_MAILER setting. The design of MAS has an idiosyncrasy where once a user (event registrant) has been registered, the MAS_MAILER that was set at that point is the one that is used for that user - it gets saved to a file, and it can't be edited. So for that "user" the mailer can't be changed. The way to get around this when you change mailers is to delete the original user and add a new user while running MAS with the MAS_MAILER variable set to the new mailer, in this case to the path of your script.

I'm running into problems with MAS not using the MAS_DIRECTORY variable. I added: MAS_DIRECTORY=/home/masuser/masconfig/ to my .profile. But MAS doesn't use the indicators and other settings files I've set up in that directory. What am I doing wrong?

Most likely you did not export the MAS_DIRECTORY environment variable. See the question above about MAS not using the mailer set up with MAS_MAILER for an example of how to export an environment variable.

I updated to a new version of MAS and when the server starts up it prints the following error and exits: "Retrieval of indicator library file indicators_persist failed. The file may be corrupted or you may have an incompatible version." How can I use the old indicators I have created with the new server?

This problem occurs when a new version of MAS incorporates a change to the structure of the indicator code that is not compatible with the old version. As a result, the old indicator file, indicators_persist, cannot be loaded when the new version of the server starts up. Once this occurs, the only way to use your old indicators is to keep using the old version of the server. (This problem can also occur with market analyzers and the generators_persist file.) However, there is a preventative method that can be used to overcome the problem: Whenever you create a new indicator, start the server in the background (-b option) and use the command-line client program, macl, to talk to the server and "record" the conversation with the command:

macl -r output_file <portnumber>

where output_file is the name of the file to which you wish to save the recorded output and <portnumber> is the port number with which you started the server. Keep this file in a safe place. It would probably also be a good idea to keep it backed up.

Whenever you upgrade to a new version of MAS and get the above error message, you can restore the indicators you have created by running the following command on each of your recorded output files:

macl -i file <portnumber>

where file is the name of a file created earlier with the -r option and <portnumber> is, again, the port number with which you started the server. This solution works for both indicators and market analyzers.

Note for Windows users: Starting MAS the "normal" way, by clicking on the MAS icon, will execute the necessary step described above of starting the server in the background. However, you will still need to run the 'macl -r ...' and 'macl -i ...' commands described above from the Windows command line. Those who do not know how to do this may want to wait for a future release in which this procedure is handled automatically.

Data-related Problems

I'm not able to view daily data in the charting GUI - only periods of weekly intervals and longer are viewable.

Most likely you are not able to see daily data on the chart because the start and end dates for daily data configured in the GUI do not fall within the date range of the data you're using (for example, the last date in your data is 1/2/2002 and the start date is configured as 12/1/2002). To fix this, either update your data so that it includes data up to the current date (or last trading day) or change the start and end date settings for daily data in the GUI.

When I try to download data off the internet using the -w parameter I get a lot of error messages that look like this:
Date input value is invalid.
Error occurred after record 871 (with date/time 05/12/2003 12:00:00.0 AM).
Discarding corrupted record.

Since the yahoo data has an open-price field (as I suppose all data sources do these days), you need to add the -o option when you start the MAS server. (The upcoming release defaults to using open fields, so the -o will not longer be needed.) Also, the data is saved in comma-separated format, so you also need to include the option -f ',' to specify that the field separator is a comma. (Tip: If someone has data in a different format, such as separated by semi-colons, you can get MAS to read the data with the option -f ';'. [The quotes are needed with UNIX/Linux to protect the semi-colon from being eaten by the shell.])

So the easiest way to start it for yahoo data would be something like:

mas -f , -o -w 5555

(where 5555 is the port number and should be replaced by the port number you are using).

I'm getting the following error from the server:

Error occurred while reading /home/user/data/x.txt: I/O error.
Error encountered - obtaining data for GUI client:
Routine failure.
Fatal error: Error occurred retrieving trading period types for x.
Exiting ...

Everything looks OK to me - the permissions on the file and the directory that contains it are correct; the content and format of x.txt looks correct. What is going wrong?

The problem is most likely that the data in x.txt is reversed. Some programs output data in the opposite order that MAS expects, with the latest date at the beginning of the file and the earliest date at the end. MAS wants the latest date at the end and the earliest date at the beginning. You can use the following script to reverse the order of the lines in a data file:

#!/usr/bin/env bash
# Reverse the order of lines in infile and output the result to outfile.

pname=$(basename $0)

usage() {
    echo "Usage: $pname infile outfile"
}

if [ $# -ne 2 ]; then
    usage; exit 1
fi

awk '{
    lines[i++] = $0
} END {
    for (j = i - 1; j >= 0; --j) {
        print lines[j]
    }
}' $1 > $2

The MAS Charting GUI

I'm trying to run MAS on a Linux system. I got the server running OK, but the GUI charting client refuses to come up. I get the error message:

java.lang.NullPointerException
   at java.awt.Menu.add(Menu.java:39)
   at mas_gui.MA_MenuBar.<init>(MA_MenuBar.java:98)
   at mas_gui.Chart.initialize_GUI_components(Chart.java:462)
   at mas_gui.Chart.<init>(Chart.java:120)
   at MA_Client.main(MA_Client.java:12)

I am using the Kaffe implementation of the Java Virtual Machine (JVM).

This is very likely the same problem that other Linux users have encountered who are using the Kaffe JVM. This implementation apparently has a bug that causes this "null pointer exception". Aside from contacting kaffe.org and convincing them to fix the problem, the easiest solution is probably to install and use an alternative JVM, such as Sun's or IBM's JVM.

I got a really weird error from the charting GUI:
"NaN encountered - substituting 0."
It didn't seem to have any effect on processing.

What is happening here is that the server is producing NaN (Not a Number) as a value, probably because of some impossible calculation, such as 3.5 / 0, which can happen inadvertently for some indicators. The GUI doesn't know how to display a NaN as a value, so it converts it to 0; but it tells you about it so that you will know that the displayed value is not really correct. Since this phenomenon is unusual, but not totally unexpected, you can regard this as an informational message rather than an error message.

I tried adding a new indicator to MAS. It worked in the command prompt and is showing in the list of indicators. But when I try to load the Java Charts the new indicator is not there in the list of indicators. I followed the instructions in the creating_indicators.txt file. Do I need to do something extra for the new indicator to show up in the java charts?

Yes, there is one more step you need to get the new indicator to show up in the charting GUI. You need to edit the .ma_clientrc file and add a line in that file for the new indicator. This file resides in the lib/classes directory, under the main MAS directory - the directory in which you installed MAS. This is not very user friendly and the GUI will probably be enhanced in the future with a feature to make the task of adding a new indicator much easier. But for now it must be done by hand.

To add the new indicator, you need to find a 3-line comment in the .ma_clientrc file that begins:

# Specify which indicators to include in the selection list, the order ...

After the comment are specifications for indicators that the GUI will request from the server. The easiest way to add a line for your new indicator is to copy an existing line and change it to fit your new indicator. For example, for a Force Index indicator (which you want to appear in the lower chart rather than the main chart) you could copy and change the line:

lower_indicator MACD Difference red

to:

lower_indicator Force Index red

(Change lower_indicator to upper_indicator if you want the indicator to appear in the main chart.) If you want the indicator to be drawn in a color other than red, pick one of the colors listed in the comment lines that begin:

# color settings - Valid colors are:

and change the string "red" to the string for the color you picked. There are two things to note here: First, the format of an indicator specification line is:

<indicator-location>{tab}<indicator-name>{tab}<indicator-color>

where:

<indicator-location> is either lower_indicator or upper_indicator
{tab} is a tab character
<indicator-name> is the name of the indicator
<indicator-color> is the color of the indicator

The second thing is that the indicator name has to match exactly the name you gave the indicator when you created it with the MAS program.

One more obstacle you may encounter is that the file is in UNIX text format, which means that many Windows-based editors (since they require both a newline and linefeed character to indicate a new line) will display the file as one long line. It will be very difficult to edit the file with such an editor. To solve this problem, you can obtain an editor that recognizes UNIX format. (I haven't researched it, but I would guess that there are several such editors available for free on the web. Vim, a VI clone, is one, and, most likely, a Windows version of emacs exists that can also do this. A quick search on google.com unearthed this site, which lists several Windows editors. One of the links on that site leads to a free Windows editor that appears to handle UNIX text files.) I'm aware that dealing with this obstacle will be a major inconvenience for some people. This is one reason I'm hoping to be able to add a feature to the GUI to make it easy to add a new indicator.

Database Problems

I installed MAS and when I try to run it, I get the following error [or something similar]:

mas: error while loading shared libraries: libodbc.so.1: cannot open shared object file: No such file or directory

See Administration and Setup for the solution to this problem.

I get this error when I start MAS using my odbc database connection: "Error occurred retrieving non-intraday period types for a" (a being the first stock symbol on my list). (This answer covers some other SQL errors as well.)

Possible solutions are as follows:

I wish to set MAS up to work with a database with ODBC. I am trying to create the needed database tables for MAS and from the sysadmin docs it is mentioned that there are Postgres scripts for doing so. However, I have looked and looked in the files that were installed with MAS but have not found them.

The example script files for creating MAS tables reside in a file separate from the main MAS installation file. You will find them in the scripts archive file in the download section - either scripts{relnum}.tar.gz or scripts{relnum}.zip -, where {relnum} stands for the release number - e.g., 1.6.5. (The scripts files for the latest release, 1.6.5 - as of June 2003 - are scripts1.6.5.tar.gz and scripts1.6.5.zip.) Several people have asked me this question, so I've concluded that I need to include the database scripts with the main installation file. I will probably do this for the next official (non-beta) release.

Technical Analysis Indicators

How do I change the parameter settings of a technical indicator? (For example, how would I change the parameter settings of "Slow Stochastic %D" from 5, 3, 3, to 11, 9, 8?)

In the current version of MAS, you can change the parameter settings of an indicator by starting up the MAS command-line client (either macl or maclj) and entering 'e' (Edit indicators) and then 'e' (Edit an indicator) again. You will see a list of all available indicators. Select the indicator you want to edit by entering the number of the indicator from the list. You will see a list of the editable parameters for the indicator. Select the parameter you want to edit by entering the number of the parameter from the list. Enter the new value you wish to assign for that parameter. Continue editing the parameters in this way until you've changed all that you wish to change. Enter '0' (zero) to leave the change-parameter menu. Enter '0' again to leave the edit-indicator menu. Enter 's' (Save changes) to save the changes you have just made. Enter '-' (Previous) to leave the edit-indicators menu. Enter 'x' to exit the command-line client.

I tried adding a new indicator to MAS. It worked in the command prompt and is showing in the list of indicators. But when I try to load the Java Charts the new indicator is not there in the list of indicators. I followed the instructions in the creating_indicators.txt file. Do I need to do something extra for the new indicator to show up in the java charts?

This question is covered in the MAS Charting GUI section.

Signal Generation and Market Analyzers

I haven't had much luck with the log file for sending the market analysis - it produces a core dump when I try and do this.

File logging is broken in older versions of MAS. However, it appears to be fixed in the 1.6.5 version and later. My tests of file logging with these later versions of MAS were successful, but I have not tested it extensively. If anyone has a problem with using a log file for market analysis (storing trading signals in a file), please let me know.

Is there anyway to get buy/sell signals without emailing them to someone - for example, just listing them to STDOUT. This will allow me to use the data how I would like in perl or what ever, instead of perl going to my email and parsing the data.

You can do this either by using the file logging mechanism for delivering signals (by choosing "Log file" instead of "User" for the registrant type), or by replacing the MAS "mailer" program with a script to do whatever you want with the signals sent from MAS. For the latter solution, just write the script to do whatever you want with the data, parsing it according to the format that shows up in email messages when MAS uses a regular mailer. Once you write the script, you can tell MAS to use it as your mailer by setting the MAS_MAILER environment variable (documented in the file env_settings) to the path of the script, making sure it is executable, of course. Then use the event registration feature of MAS (from the command-line client) to register a new user that will receive signals using your script. Here's a simple example:

#!/bin/bash

mas_dir=/path_where_mas_signals_are_stored
/bin/cat >> $mas_dir/events$(/bin/date +%Y%m%d)

Of course, you can do more sophisticated filtering, such as putting buy and sell signals into separate files according to the subject line. I've written a Python script that adds or removes stock symbols from watchlists according to whether a signal arrives indicating that the stock is in a long-term uptrend, downtrend, or is going sideways.

When building compound generators (market analyzers) are the links to the lower level generators static or dynamic? For example, if I build an RSI generator and use it in a further generator do changes to the original RSI generator affect the compound generator - is it statically or dynamically linked?

It is, to use your words, dynamic. A more precise way of putting it is that a when a compound generator uses an existing generator it gets a copy of that generator rather than "linking" to the original. Thus if you change the original generator, the compound generator using a copy of the original is not affected. This is also true of changes in the other direction: If you change a sub-generator of a compound generator, the original from which the sub-generator was copied is not affected. For example (using the generators that come with the current release of MAS), if you change the "MACD Crossover and Stochastic %D Crossover (Buy)" (a compound indicator that uses "MACD Crossover (Buy)" and "Stochastic %D Crossover (Buy)") such that you change the parameters of its MACD and its stochastic components (and save the changes), and then use the "market analyzer" editing function to view the parameters for the "MACD Crossover (Buy)" and "Stochastic %D Crossover (Buy)" generators you will see that their values have not changed.

Having studied the doc Creating Market Analyzers ..... I have run into a brick wall! Using your diagram convention what I want to do is:

    GT_OPERATOR
       / \
      /   \
    RSI   Constant

The question is: How do I get RSI as the left Operand? I have played with most of the really obvious and not so obvious selections and have drawn a blank!

Either the basic linear command or the basic numeric command will work here. So, using the basic linear command, your diagram would change to (where BLC is the basic linear command):

    GT_OPERATOR
       / \
      /   \
    BLC   Constant

Note: In the latest version of mas, Constant has been renamed as Numeric-value-command. (Since it can function as either a constant or as a variable whose value can change during a run.)

I have been trying to get MAS to give me MACD data that matches BigCharts. They both use MACD values of 26, 12, & 9. MAS defaults, when installed, to 13, 5, & 6. I modified all of the MACD analyzer values to long = 26, short = 12, and average = 9, to match BigCharts. This morning I got this signal:

Event for: BHI, date: 05/19/2003, type: MACD Crossover (Buy)
description: Crossover for weekly trading period with indicators:
MACD Difference and
MACD Signal Line (EMA of MACD Difference)
values: -0.285593, -0.325897

Looking at BigCharts, it shows that the last MACD buy crossover happened at about 3 May, not 19 May. How can MAS give me an answer so dramatically different than BigCharts?

It looks like you are using the existing MACD market analyzers that come with the MAS release. These analyzers operate on weekly data - that is, they operate on data with a weekly period type. (The phrase "Crossover for weekly trading period" provides the clue here.) Most likely you are using daily data at BigCharts. This would account for the difference that you are seeing. If you want to apply market analyzers to daily data, you'll need to use the MAS command-line to create your own analyzers and select "daily" as the period type.