Friday, February 6, 2015

Batteries included!

The newest versions of the RevitPythonShell include a copy of the python standard library. This copy is taken from IronPython. This means you don’t need to add a search path to a local python distribution when importing standard python modules.

Since the library is included in a zip file embedded in the RpsRuntime.dll, it is also automatically deployed with when creating RpsAddIns.

The versions supporting this feature are:

  • (r223) Installer for Autodesk Revit 2015 (experimental)
  • Installer for Autodesk Vasari (Beta 3)
  • List item

Thursday, January 15, 2015

Introducing the non-modal shell in RevitPythonShell (r223)

You have no idea how excited I am about this release! Revision r223 of the RevitPythonShell has been tested by a select few very brave people (Ehsan Iran Nejad and Callum Freeman) and seems to work. Normally I’m not so worried about pushing out new versions, but this one is… different.
The idea actually came from Ehsan. The Next Big Thing. Also known as the non-modal shell.
I know, I know - I need to get a lot better at coming up with good names for the various parts in RevitPythonShell, but for now, that is what it is called: Non-modal shell.
The non-modal shell is different from the standard RevitPythonShell shell in that it is… well… non-modal. What does that mean? Currently, when you start a shell in RPS, the shell gets the focus and you can’t interact with the rest of the Revit GUI until you close the shell again. This is cool for working with short scripts, trying stuff out etc, but we can do better:
The non-modal shell does not block the rest of Revit. You can select stuff in the view, make changes to the BIM etc. And then query the selection in the shell or do whatever you like.
With some exceptions.
You see, because the shell is non-modal, it is not running in the special thread Revit needs you to be in for accessing the API. You would get a lot of ugly errors, if there wasn’t some clever magic going on in the background that sends all input to the interactive shell to an implementation of IExternalEventHandler which Revit then runs in the correct thread and everything is fine and dandy.
What doesn’t work though, is using transactions in the interactive shell:
>>> t = Transaction(doc, 'test')
>>> t.Start()
>>> t.GetStatus()
>>> t.Commit()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Exception: The transaction has not been started yet (the current status is not 'Started'). A transaction must be started before it can be either committed or rolled back.
You can get around this by using the IronPython Pad - the little editor below the interactive shell. This allows you to run multiple lines at a time and those will be run in the same IExternalEventHandler so the problem does not show up.
You can find the installer for this (experimental) version of RevitPythonShell in the Downloads section of the project homepage.
I intend to post some more articles in the future that showcase some other new features in the r223 release, so you can expect some more goodies soon :)

Monday, December 22, 2014

A short tutorial on the esoreader module

This post explains how to use the esoreader, a python module for parsing the .eso file produced by EnergyPlus. It also includes a small (incomplete) reverse engineering of the .eso file format.
EnergyPlus is a whole building simulation tool EnergyPlus is a whole building energy simulation program that engineers, architects, and researchers use to model energy and water use in buildings. The .eso file is the main output file produced by EnergyPlus and is normally parsed by tools that come with the EnergyPlus suit of tools. The esoreader module lets you read in the time series data in python scripts, which for my research is quite useful. I published the module thinking other people might want to do so too.
Last week I got an email about how the documentation on the pypi page for the esoreader module is rather… terse. So I went to check the esoreader page and yes, there is not a lot of documentation. The example code I published was this:
import esoreader
PATH_TO_ESO = r'/Path/To/EnergyPlus/Output/eplusout.eso'
dd, data =
frequency, key, variable = dd.find_variable(
    'Zone Ventilation Total Heat Loss Energy')[0]
idx = dd.index[frequency, key, variable]
time_series = data[idx] 
What can I say? There is not much more you can do with esoreader. I think the best way to understand the module is to look at the .eso file format:
The eso file format starts of with a header section called the “data dictionary” (I used the variable dd in the example code for that). The first few lines of a sample eso file look something like this:
Program Version,EnergyPlus-Windows-32, YMD=2014.03.20 14:18
1,5,Environment Title[],Latitude[deg],Longitude[deg],Time Zone[],Elevation[m]
2,6,Day of Simulation[],Month[],Day of Month[],DST Indicator[1=yes 0=no],Hour[],StartMinute[],EndMinute[],DayType
3,3,Cumulative Day of Simulation[],Month[],Day of Month[],DST Indicator[1=yes 0=no],DayType  ! When Daily Report Variables Requested
4,2,Cumulative Days of Simulation[],Month[]  ! When Monthly Report Variables Requested
5,1,Cumulative Days of Simulation[] ! When Run Period Report Variables Requested
6,1,DEFAULT_ZONE,Zone Outdoor Air Drybulb Temperature [C] !TimeStep
99,1,DPVWALL:1157026,Surface Outside Face Temperature [C] !TimeStep
100,1,DPVWINDOW:COMBINED:DPVWALL:1157026:DEFAULTWINDOWCONSTRUCTION,Surface Outside Face Temperature [C] !TimeStep
101,1,DPVWALL:1157027,Surface Outside Face Temperature [C] !TimeStep
102,1,DPVWINDOW:COMBINED:DPVWALL:1157027:DEFAULTWINDOWCONSTRUCTION,Surface Outside Face Temperature [C] !TimeStep
103,1,DPVWALL:1157028,Surface Outside Face Temperature [C] !TimeStep
104,1,DPVWINDOW:COMBINED:DPVWALL:1157028:DEFAULTWINDOWCONSTRUCTION,Surface Outside Face Temperature [C] !TimeStep
105,1,DPVWALL:1157029,Surface Outside Face Temperature [C] !TimeStep
106,1,DPVWINDOW:COMBINED:DPVWALL:1157029:DEFAULTWINDOWCONSTRUCTION,Surface Outside Face Temperature [C] !TimeStep
107,1,DPVFLOOR:1157042,Surface Outside Face Temperature [C] !TimeStep
108,1,DPVROOF:1157058.0,Surface Outside Face Temperature [C] !TimeStep
109,1,DPVROOF:1157058.1,Surface Outside Face Temperature [C] !TimeStep
110,1,DPVROOF:1157058.2,Surface Outside Face Temperature [C] !TimeStep
111,1,DPVROOF:1157058.3,Surface Outside Face Temperature [C] !TimeStep
112,1,DEFAULT_ZONE,Zone Mean Air Temperature [C] !TimeStep
278,1,DEFAULT_ZONEZONEHVAC:IDEALLOADSAIRSYSTEM,Zone Ideal Loads Zone Total Heating Energy [J] !TimeStep
279,1,DEFAULT_ZONEZONEHVAC:IDEALLOADSAIRSYSTEM,Zone Ideal Loads Zone Total Cooling Energy [J] !TimeStep
End of Data Dictionary
The first line is stored in the DataDictionary object (dd) as version and timestamp. After that, each line represents a variable being reported. Each such variable has an index, a number of values being reported and then a reporting frequency. Well… the first few lines (indexes 1 through 5) are a bit special and I just discard them. The rest of the data dictionary lines are built like this:
  • index (e.g. 100)
  • column count (is always one as far as I can tell)
  • key (the same variable can be measured for different keys, as per the Output:Variable object in the IDF file)
    (e.g. “DPVWINDOW:COMBINED:DPVWALL:1157026:DEFAULTWINDOWCONSTRUCTION”, a surface name in one of my models)
  • variable name (e.g. “Surface Outisde Face Temperature”)
  • unit (e.g. “C”)
  • reporting frequency (e.g. “TimeStep”)
These get parsed into a DataDictionary object and stored in the attributes variables and index.
variables = dict of ids, int => [reporting_frequency,
                                 key, variable, unit]

index = dict {(key, variable, reporting_frequency) => id)}
here is an example (I’m using the IPython shell, in case you’re wondering about the In [71] line - check it out! it is awesome!!)
In [71]: dd.variables.items()[1]
  'Surface Outside Face Temperature',
The DataDictionary object has a method find_variable. Say, you want to find the variable for ‘Zone Mean Air Temperature’:
In [75]: dd.find_variable('Zone Mean Air Temperature')
Out[75]: [('TimeStep', 'DEFAULT_ZONE', 'Zone Mean Air Temperature')]
Notice how the result is a list? If you had looked for surface temperatures instead:
In [76]: dd.find_variable('surface')
[('TimeStep', 'DPVROOF:1157058.1', 'Surface Outside Face Temperature'),
 ('TimeStep', 'DPVWALL:1157029', 'Surface Outside Face Temperature'),
  'Surface Outside Face Temperature'),
 ('TimeStep', 'DPVWALL:1157028', 'Surface Outside Face Temperature'),
 ('TimeStep', 'DPVROOF:1157058.3', 'Surface Outside Face Temperature'),
 ('TimeStep', 'DPVROOF:1157058.0', 'Surface Outside Face Temperature'),
  'Surface Outside Face Temperature'),
  'Surface Outside Face Temperature'),
 ('TimeStep', 'DPVWALL:1157027', 'Surface Outside Face Temperature'),
  'Surface Outside Face Temperature'),
 ('TimeStep', 'DPVFLOOR:1157042', 'Surface Outside Face Temperature'),
 ('TimeStep', 'DPVWALL:1157026', 'Surface Outside Face Temperature'),
 ('TimeStep', 'DPVROOF:1157058.2', 'Surface Outside Face Temperature')]
you’d have gotten a list of all variables that match ‘surface’ (case-insensitive, substring match). The tuples define the variable you’re looking for: Frequency, key and variable name, since you can have the same variable output for different frequencies and keys!
So the index of the variable we’re looking for (Zone Mean Air Temperature) can be found like this:
In [77]: dd.index['TimeStep', 'DEFAULT_ZONE', 'Zone Mean Air Temperature']
Out[77]: 112
When you parse an .eso file, you get two values back: The DataDictionary and the data itself, which is stored in a simple dictionary mapping the variable index to the timeseries data:
In [82]: dd, data ='RevitToCitySim_fmibeta.eso')

In [84]: data[112]
Where does that data come from? From the rest of the .eso file, which looks like this:
1,Zuerich-SMA - - TMY2-66600 WMO#=,  47.38,   8.57,   1.00, 556.00
2,1, 1, 1, 0, 1, 0.00,60.00,Tuesday        
2,1, 1, 1, 0, 2, 0.00,60.00,Tuesday        
2,1, 1, 1, 0, 3, 0.00,60.00,Tuesday        
For all the main variables (id > 5) the format is:
  • index
  • value
Thus, the data dictionary is necessary to figure out what variables (with what frequency) are being output.
To sum up the tutorial: The code on the pypi page shows you pretty much all you can do and also all you need to do to retrieve a specific timeseries from an .eso file:
  • read in the eso file to obtain the data dictionary and the data
  • find the key, frequency and variable name you need in the data dictionary (with find_variable) or by guessing from your IDF input
  • retrieve the index of that variable
  • retrieve the time series data using that index

Thursday, October 17, 2013

How to replace a color in a PNG with python preserving transparency

Here is a small bit of code I thought I'd share with you, since I couldn't find a good solution on the web quickly...

I was coding up a web site with a horizontal navigation menu that used a downwards pointing arrow to denote a menu entry that has a drop down menu. The graphics designer had given me the image for this arrow. Here it is enlarged in a GIMP window:

Which is all fine and dandy, except, when the mouse hovers above the menu item, the arrow should turn cyan. #00aeef.

I was just about to ask the designer for the cyan version of the arrow, after failing to find a quick and easy way to change all the white pixels to cyan preserving transparency, when I decided to see how hard that would be to do with the Python Imaging Library (PIL).

This is the throwaway script I came up with:

OLD_PATH = r'c:\path\to\images\arrow_white.png'
NEW_PATH = r'c:\path\to\images\arrow_cyan.png'

R_OLD, G_OLD, B_OLD = (255, 255, 255)
R_NEW, G_NEW, B_NEW = (0, 174, 239)

import Image
im =
pixels = im.load()

width, height = im.size
for x in range(width):
    for y in range(height):
        r, g, b, a = pixels[x, y]
        if (r, g, b) == (R_OLD, G_OLD, B_OLD):
            pixels[x, y] = (R_NEW, G_NEW, B_NEW, a)

As you can see, that was not so difficult at all... And produced this result:

Let's just go through this quickly... OLD_PATH and NEW_PATH should be self explaining.

Next, I configure the R(ed), G(reen) and B(lue) values of the color I'm looking for and the color I want to change that to. White is easy, you should be able to do that in your head, but for the cyan bit, the python interpreter can help us:

>>> int('00', 16)
>>> int('ae', 16)
>>> int('ef', 16)

Converting bytes from hex to decimal is equivalent to parsing the hex string as an int with base 16. The other way round is even easier:

>>> hex(0)
>>> hex(174)
>>> hex(239)

Next, I import the imaging library and open the image. The load() method returns a map of the image, which each x,y-coordinate denoting a tuple (r, g, b, a) with a being the alpha channel or transparency.

The rest of the script is therefor just a simple iteration over the width and height of the image, pixel by pixel, testing for the old color and replacing that with the new color if found, preserving the original transparency.

Thursday, May 30, 2013

New Feature: StartupScript in RevitPythonShell

Here is a new hidden feature in recent versions of the RevitPythonShell: You can specify a script to start when Revit starts. The script is called during the IExternalApplication.OnStartup event when all the Revit plugins get to do their initialization.

This feature is still not quite official yet, but I have been using it for various purposes internally and think it works quite nicely. There are some changes between the code I have locally and what you probably have installed - I will point these out below.

To specify a StartupScript for RevitPythonShell, add a tag called StartupScript to the RevitPythonShell.xml file at the same level as the Commands, Variables, SearchPaths and InitScript tags. This tag has a single attribute src that specifies the path to the script to run.

You can find the RevitPythonShell.xml file in the folder %APPDATA%\RevitPythonShell2013.

This feature is also present in deployed RpsAddins!


<?xml version="1.0" encoding="UTF-8"?>
  <StartupScript src="c:\path-to-your-script\"/>
    <Command name="Hello World" src="C:\...\" group="Examples"/>
    <!-- ... -->


If you are not building from source, then the attribute to use in the StartupScript tag is still source. I changed it to src for consistency with the Command tag. Future versions of RevitPythonShell (especially the 2014 line) will use src as described in this post!

There is currently no GUI for changing the StartupScript. Also, this is somewhat of a specialist feature: You probably don't want to use it except for some very special cases.

So... what can you do with a startup script? So far, I have used it for two projects...

In the first project, I load a web server and hook it up to the Idling event so that I can provide an HTTP interface to Revit. My day job involves extracting an abstract model of simulation relevant aspects of a building from Revit, much like gbXML and being able to do that from outside Revit gives me some flexibility.

In the second project, I was automating a solar study: For each window in an elaborate model, I traced rays from a grid on the window to the sun from the SunAndShadowSettings to determine the percentage of direct sunlight a window gets - for every 15 minutes in a whole year! Since the model was big, with a lot of neighboring buildings modeled as mass objects, ray tracing (using the FindReferencesByDirectionWithContext method) gets very slow. Simulating a single day takes about 20 minutes. And simulating the whole year just broke down with Revit crashing. So... I decided to do each day separatly, and just start Revit after each day - the startup script would check the __vars__[START_DAY] variable to get the current day to simulate and on successful simulation, increment that day (this is where the writeable __vars__ feature was born) and quit Revit. Then I wrapped that up in a simple script that just started looped 365 times, starting Revit and waiting for it to finish.

Side note: quitting Revit is not that easy. Or really easy, whichever way you want to view it. I ended up using this code to force the process to die:

from System.Diagnostics import Process
p = Process.GetCurrentProcess()

Wednesday, May 29, 2013

__vars__ is now writeable

Here is another feature I sneaked in without documentation: The __vars__ dictionary is now writeable, that is, assigning to a key in the dictionary saves the variable to the RevitPythonShell.xml file.

This feature can be used for storing data in between invocations of a script or even Revit sessions!


Changing the RevitPythonShell.xml file manually will not be reflected in the __vars__ dictionary until the next invocation of the shell.

The implementation for the writeable dictionary is in RevitPythonShell.RpsRuntime.SettingsDictionary, which implements IDictionary<string, string>.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml.Linq;

namespace RevitPythonShell.RpsRuntime
    /// <summary>
    /// A subclass of Dictionary<string, string>, that writes changes back to a settings xml file.
    /// </summary>
    public class SettingsDictionary : IDictionary<string, string>
        private readonly IDictionary<string, string> _dict;
        private readonly string _settingsPath;
        private XDocument _settings;

        public SettingsDictionary(string settingsPath)
            _settingsPath = settingsPath;
            _settings = XDocument.Load(_settingsPath);

            _dict = _settings.Root.Descendants("StringVariable").ToDictionary(
                v => v.Attribute("name").Value,
                v => v.Attribute("value").Value);

        private void SetVariable(string name, string value)
            var variable = _settings.Root.Descendants("StringVariable").Where(x => x.Attribute("name").Value == name).FirstOrDefault();
            if (variable != null)
                variable.Attribute("value").Value = value.ToString();
                    new XElement("StringVariable", new XAttribute("name", name), new XAttribute("value", value)));

        private void RemoveVariable(string name)
            var variable = _settings.Root.Descendants("StringVariable").Where(x => x.Attribute("name").Value == name).FirstOrDefault();
            if (variable != null)

        private void ClearVariables()
            var variables = _settings.Root.Descendants("StringVariable");
            foreach (var variable in variables)

        public void Add(string key, string value)
            _dict.Add(key, value);
            SetVariable(key, value);

        public bool ContainsKey(string key)
            return _dict.ContainsKey(key);

        public ICollection<string> Keys
            get { return _dict.Keys; }

        public bool Remove(string key)
            return _dict.Remove(key);

        public bool TryGetValue(string key, out string value)
            return _dict.TryGetValue(key, out value);

        public ICollection<string> Values
            get { return _dict.Values; }

        public string this[string key]
                return _dict[key];
                _dict[key] = value;
                SetVariable(key, value);

        public void Add(KeyValuePair<string, string> item)
            SetVariable(item.Key, item.Value);

        public void Clear()

        public bool Contains(KeyValuePair<string, string> item)
            return _dict.Contains(item);

        public void CopyTo(KeyValuePair<string, string>[] array, int arrayIndex)
            _dict.CopyTo(array, arrayIndex);

        public int Count
            get { return _dict.Count; }

        public bool IsReadOnly
            get { return false; }

        public bool Remove(KeyValuePair<string, string> item)
            return _dict.Remove(item);

        public IEnumerator<KeyValuePair<string, string>> GetEnumerator()
            return _dict.GetEnumerator();

        System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
            return _dict.GetEnumerator();

Tuesday, May 21, 2013

Deploying RPS scripts with DeployRpsAddin

I know nobody is checking the code I check into the RevitPythonShell code repository, because nobody bothered to ask me about the new killer feature i sneaked in. For a project at work that I used the RevitPythonShell for, the need arose to deploy a bunch of scripts on a customers computer. Now, RPS itself has a simple installer that you can execute and go next, next, next, finish and voilĂ , you have RPS installed on your computer. But... any scripts you write need to be copied to the customers computer, then RPS has to be installed and configured to pick up the script. Oh, and don't forget to install cpython and any modules you make use of.

Have you tried writing instructions for such a manual installation procedure? I have. It is not fun at all! I have also tried to get people to execute these instructions. It just doesn't fly.

This is where the new feature of RPS comes in: In the Ribbon, there is now an additional button called "Deploy RpsAddin" that is located under the button for starting the shell, just above the one for customizing RPS.

Deploying an RPS script as a standalone addin involves choosing an xml file that acts as a manifest or package description of the addin you are about to create. Basically, you describe which python scripts to assign to which buttons and it creates a new dll for you that includes all these scripts and can be used as an addin in Revit. You just need to write an addin manifest and place it in the right position.

The source comes with an example addin called HelloWorld. That really is all it does: a Button that prints "Hello, World!" to the screen. But I include the xml deployment file and also an InnoSetup script to get you started on deploying your own addins.

When you want to include a cpython library, you will need to make sure that this is also in the search path of the addin's bundled interpreter. So, the addin includes a bunch of files, dlls, a version of IronPython (2.7) and also the new RpsRuntime dll that handles the parts of RPS that get used by both the standard RPS version and deployed addins.

You can include cpython modules in your setup program, copying them for instance into your installation directory and then go from there. There is an equivalent of the RevitPythonShell2013.xml file that gets deployed with your addin to the %AppData% folder that you can use for setting up stuff.

The structure of the deployment xml file looks like this: (I will call it RpsAddin xml file from now on)

<?xml version=" 1.0" encoding=" utf-8" ?>
  <RibbonPanel text=" Hello World">
    <!-- the script is always searched relative to the location of the RpsAddin xml file -->
    < PushButton text ="Hello World! " script =" "/>

You can add as many RibbonPanel tags as you would like. Each PushButton is then placed on that panel and assigned to the script. The path to the script is relative to the RpsAddin xml file. The DLL that gets created is placed in a folder "Output_YOUR_RPSADDIN_NAME" relative to the RpsAddin xml. The name of your addin is taken from the name you call the RpsAddin xml file. In this case, the file is called "HelloWorld.xml", so the Addin will be called "HelloWorld", a folder "Output_HelloWorld" is created with the dll "HelloWorld.dll", "RpsRuntime.dll" and a bunch of IronPython dlls.

You can then use an InnoSetup file to create an installer for this. The HelloWorld example comes with this file:

Source: Output_HelloWorld\RpsRuntime.dll; DestDir: {app};
Source: Output_HelloWorld\IronPython.dll; DestDir: {app};
Source: Output_HelloWorld\IronPython.Modules.dll; DestDir: {app};
Source: Output_HelloWorld\Microsoft.Scripting.Metadata.dll; DestDir: {app};
Source: Output_HelloWorld\Microsoft.Dynamic.dll; DestDir: {app};
Source: Output_HelloWorld\Microsoft.Scripting.dll; DestDir: {app};

; this is the main dll with the script embedded
Source: Output_HelloWorld\HelloWorld.dll; DestDir: {app};

; add a similar line, if your addin requires a configuration file (search paths or predefined variables)
;Source: HelloWorld.xml; DestDir: {userappdata}\HelloWorld; Flags: onlyifdoesntexist;

{ install revit manifest file }
procedure CurStepChanged(CurStep: TSetupStep);
  AddInFilePath: String;
  AddInFileContents: String;

  if CurStep = ssPostInstall then

  { GET LOCATION OF USER AppData (Roaming) }
  AddInFilePath := ExpandConstant('{userappdata}\Autodesk\Revit\Addins\2013\HelloWorld.addin');

  AddInFileContents := '<?xml version="1.0" encoding="utf-8" standalone="no"?>' + #13#10;
  AddInFileContents := AddInFileContents + '<RevitAddIns>' + #13#10;
  AddInFileContents := AddInFileContents + '  <AddIn Type="Application">' + #13#10;
    AddInFileContents := AddInFileContents + '    <Name>HelloWorld</Name>' + #13#10;
  AddInFileContents := AddInFileContents + '    <Assembly>'  + ExpandConstant('{app}') + '\HelloWorld.dll</Assembly>' + #13#10;

  { NOTE: create your own GUID here!!! }
  AddInFileContents := AddInFileContents + '    <AddInId>276D41F2-CCC4-4B55-AF2A-47D30227F289</AddInId>' + #13#10;

  AddInFileContents := AddInFileContents + '    <FullClassName>HelloWorld</FullClassName>' + #13#10;

  { NOTE: you should register your own VendorId with Autodesk }
  AddInFileContents := AddInFileContents + '  <VendorId>RIPS</VendorId>' + #13#10;
  AddInFileContents := AddInFileContents + '  </AddIn>' + #13#10;
  AddInFileContents := AddInFileContents + '</RevitAddIns>' + #13#10;
  SaveStringToFile(AddInFilePath, AddInFileContents, False);


{ uninstall revit addin manifest }
procedure CurUninstallStepChanged(CurUninstallStep: TUninstallStep);
  AddInFilePath: String;
  if CurUninstallStep = usPostUninstall then
    AddInFilePath := ExpandConstant('{userappdata}\Autodesk\Revit\Addins\2013\HelloWorld.addin');

    if FileExists(AddInFilePath) then


I included some Pascal code for installing an addin manifest to the %APPDATA% folder. This is generally something like C:\Users\username\AppData\Roaming\ADDIN_NAME. Running this setup will produce a file called Setup_Helloworld.exe that can then be given to your friends to try out your new cool HelloWorld Revit Addin, coded in the sweet python language we all love so much!