Accessing specific overloads in IronPython (Document.LoadFamily)

The RevitPythonShell makes working with Revit a bit easier, because, you know, Python. The specific flavour of Python used is IronPython - a port of the Python language to the .NET platform. IronPython lets you call into .NET objects seamlessly. In theory. Except when it doesen’t. All abstractions are leaky.
This article is all about a specific leak, based on an impedance mismatch between the object model used in .NET and that used in Python: In C#, you can overload a method. A simple way to think about this is to realize that the name of the method includes its signature, the list of parameter (types) it takes. Go read a book if you want the gory details. Any book. I’m just going to get down to earth here and talk about a specific example:
The Document.LoadFamily method.
The standard method for selecting a specific overload is just calling the function and having IronPython figure it out.
I guess the place to read up on how to call overloaded methods is here: http://ironpython.net/documentation/dotnet/dotnet.html#method-overloads. To quote:

When IronPython code calls an overloaded method, IronPython tries to select one of the overloads at runtime based on the number and type of arguments passed to the method, and also names of any keyword arguments.

This works really well if the types passed in match the signatures of a specific method overload well. IronPython will try to automatically convert types, but will fail with a TypeError if more than one method overload matches.
The Document.LoadFamily method is special in that one of its parameters is marked as out in .NET - according to the standard IronPython documentation (REF) that should translate into a tuple of return values - and it does, if you know how. It is just non-intuitive - see this question on Stack Overflow:

revitpythonshell provides two very similar methods to load a family.

LoadFamily(self: Document, filename:str) -> (bool, Family)
LoadFamily(self: Document, filename:str) -> bool

So it seems like only the return values are different. I have tried to calling it in several different ways:
(success, newFamily) = doc.LoadFamily(path)
success, newFamily = doc.LoadFamily(path)
o = doc.LoadFamily(path)
But I always just get a bool back. I want the Family too.

What is happening here is that the c# definitions of the method are:

public bool LoadFamily(
    string filename
)

and

public bool LoadFamily(
    string filename,
    out Family family
)

The IronPython syntax candy for out parameters, returning a tuple of results, can’t automatically be selected here, because calling LoadFamily with just a string argument matches the first method overload.
You can get at the overload you are looking for like this:

import clr
family = clr.Reference[Family]()
# family is now an Object reference (not set to an instance of an object!)
success = doc.LoadFamily(path, family)  # explicitly choose the overload
# family is now a Revit Family object and can be used as you wish

This works by creating an object reference to pass into the function and the method overload resultion thingy now knows which one to look for.
Working under the assumption that the list of overloads shown in the RPS help is the same order as they appear, you can also do this:

success, family = doc.LoadFamily.Overloads.Functions[0](path)

and that will, indeed, return a tuple (bool, Autodesk.Revit.DB.Family). I just don’t think you should be doing it that way, as it introduces a dependency on the order of the method overloads - I wouldn’t want that smell in my code…
Note, that this has to happen inside a transaction, so a complete example might be:

import clr
t = Transaction(doc, 'loadfamily')
t.Start()
try:
    family = clr.Reference[Family]()
    success = doc.LoadFamily(path, family)
    # do stuff with the family
    t.Commit()
except:
    t.Rollback()

Introducing the non-modal shell in RevitPythonShell (r223)

You have no idea how excited I am about this release! Revision r223 of the RevitPythonShell has been tested by a select few very brave people (Ehsan Iran Nejad and Callum Freeman) and seems to work. Normally I’m not so worried about pushing out new versions, but this one is… different.
The idea actually came from Ehsan. The Next Big Thing. Also known as the non-modal shell.
I know, I know - I need to get a lot better at coming up with good names for the various parts in RevitPythonShell, but for now, that is what it is called: Non-modal shell.
The non-modal shell is different from the standard RevitPythonShell shell in that it is… well… non-modal. What does that mean? Currently, when you start a shell in RPS, the shell gets the focus and you can’t interact with the rest of the Revit GUI until you close the shell again. This is cool for working with short scripts, trying stuff out etc, but we can do better:
The non-modal shell does not block the rest of Revit. You can select stuff in the view, make changes to the BIM etc. And then query the selection in the shell or do whatever you like.
With some exceptions.
You see, because the shell is non-modal, it is not running in the special thread Revit needs you to be in for accessing the API. You would get a lot of ugly errors, if there wasn’t some clever magic going on in the background that sends all input to the interactive shell to an implementation of IExternalEventHandler which Revit then runs in the correct thread and everything is fine and dandy.
What doesn’t work though, is using transactions in the interactive shell:

>>> t = Transaction(doc, 'test')
>>> t.Start()
Autodesk.Revit.DB.TransactionStatus.Started
>>> t.GetStatus()
Autodesk.Revit.DB.TransactionStatus.RolledBack
>>> t.Commit()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Exception: The transaction has not been started yet (the current status is not 'Started'). A transaction must be started before it can be either committed or rolled back.
>>>

You can get around this by using the IronPython Pad - the little editor below the interactive shell. This allows you to run multiple lines at a time and those will be run in the same IExternalEventHandler so the problem does not show up.
You can find the installer for this (experimental) version of RevitPythonShell in the Downloads section of the project homepage.
I intend to post some more articles in the future that showcase some other new features in the r223 release, so you can expect some more goodies soon :)

A short tutorial on the esoreader module

This post explains how to use the esoreader, a python module for parsing the .eso file produced by EnergyPlus. It also includes a small (incomplete) reverse engineering of the .eso file format.
EnergyPlus is a whole building simulation tool EnergyPlus is a whole building energy simulation program that engineers, architects, and researchers use to model energy and water use in buildings. The .eso file is the main output file produced by EnergyPlus and is normally parsed by tools that come with the EnergyPlus suit of tools. The esoreader module lets you read in the time series data in python scripts, which for my research is quite useful. I published the module thinking other people might want to do so too.
Last week I got an email about how the documentation on the pypi page for the esoreader module is rather… terse. So I went to check the esoreader page and yes, there is not a lot of documentation. The example code I published was this:

import esoreader
PATH_TO_ESO = r'/Path/To/EnergyPlus/Output/eplusout.eso'
dd, data = esoreader.read(PATH_TO_ESO)
frequency, key, variable = dd.find_variable(
    'Zone Ventilation Total Heat Loss Energy')[0]
idx = dd.index[frequency, key, variable]
time_series = data[idx] 

What can I say? There is not much more you can do with esoreader. I think the best way to understand the module is to look at the .eso file format:
The eso file format starts of with a header section called the “data dictionary” (I used the variable dd in the example code for that). The first few lines of a sample eso file look something like this:

Program Version,EnergyPlus-Windows-32 8.1.0.009, YMD=2014.03.20 14:18
1,5,Environment Title[],Latitude[deg],Longitude[deg],Time Zone[],Elevation[m]
2,6,Day of Simulation[],Month[],Day of Month[],DST Indicator[1=yes 0=no],Hour[],StartMinute[],EndMinute[],DayType
3,3,Cumulative Day of Simulation[],Month[],Day of Month[],DST Indicator[1=yes 0=no],DayType  ! When Daily Report Variables Requested
4,2,Cumulative Days of Simulation[],Month[]  ! When Monthly Report Variables Requested
5,1,Cumulative Days of Simulation[] ! When Run Period Report Variables Requested
6,1,DEFAULT_ZONE,Zone Outdoor Air Drybulb Temperature [C] !TimeStep
99,1,DPVWALL:1157026,Surface Outside Face Temperature [C] !TimeStep
100,1,DPVWINDOW:COMBINED:DPVWALL:1157026:DEFAULTWINDOWCONSTRUCTION,Surface Outside Face Temperature [C] !TimeStep
101,1,DPVWALL:1157027,Surface Outside Face Temperature [C] !TimeStep
102,1,DPVWINDOW:COMBINED:DPVWALL:1157027:DEFAULTWINDOWCONSTRUCTION,Surface Outside Face Temperature [C] !TimeStep
103,1,DPVWALL:1157028,Surface Outside Face Temperature [C] !TimeStep
104,1,DPVWINDOW:COMBINED:DPVWALL:1157028:DEFAULTWINDOWCONSTRUCTION,Surface Outside Face Temperature [C] !TimeStep
105,1,DPVWALL:1157029,Surface Outside Face Temperature [C] !TimeStep
106,1,DPVWINDOW:COMBINED:DPVWALL:1157029:DEFAULTWINDOWCONSTRUCTION,Surface Outside Face Temperature [C] !TimeStep
107,1,DPVFLOOR:1157042,Surface Outside Face Temperature [C] !TimeStep
108,1,DPVROOF:1157058.0,Surface Outside Face Temperature [C] !TimeStep
109,1,DPVROOF:1157058.1,Surface Outside Face Temperature [C] !TimeStep
110,1,DPVROOF:1157058.2,Surface Outside Face Temperature [C] !TimeStep
111,1,DPVROOF:1157058.3,Surface Outside Face Temperature [C] !TimeStep
112,1,DEFAULT_ZONE,Zone Mean Air Temperature [C] !TimeStep
278,1,DEFAULT_ZONEZONEHVAC:IDEALLOADSAIRSYSTEM,Zone Ideal Loads Zone Total Heating Energy [J] !TimeStep
279,1,DEFAULT_ZONEZONEHVAC:IDEALLOADSAIRSYSTEM,Zone Ideal Loads Zone Total Cooling Energy [J] !TimeStep
End of Data Dictionary

The first line is stored in the DataDictionary object (dd) as version and timestamp. After that, each line represents a variable being reported. Each such variable has an index, a number of values being reported and then a reporting frequency. Well… the first few lines (indexes 1 through 5) are a bit special and I just discard them. The rest of the data dictionary lines are built like this:

• index (e.g. 100)
• column count (is always one as far as I can tell)
• key (the same variable can be measured for different keys, as per the Output:Variable object in the IDF file)
  (e.g. “DPVWINDOW:COMBINED:DPVWALL:1157026:DEFAULTWINDOWCONSTRUCTION”, a surface name in one of my models)
• variable name (e.g. “Surface Outisde Face Temperature”)
• unit (e.g. “C”)
• reporting frequency (e.g. “TimeStep”)

These get parsed into a DataDictionary object and stored in the attributes variables and index.

variables = dict of ids, int => [reporting_frequency,
                                 key, variable, unit]

index = dict {(key, variable, reporting_frequency) => id)}

here is an example (I’m using the IPython shell, in case you’re wondering about the In [71] line - check it out! it is awesome!!)

In [71]: dd.variables.items()[1]
Out[71]:
(100,
 ['TimeStep',
  'DPVWINDOW:COMBINED:DPVWALL:1157026:DEFAULTWINDOWCONSTRUCTION',
  'Surface Outside Face Temperature',
  'C'])

The DataDictionary object has a method find_variable. Say, you want to find the variable for ‘Zone Mean Air Temperature’:

In [75]: dd.find_variable('Zone Mean Air Temperature')
Out[75]: [('TimeStep', 'DEFAULT_ZONE', 'Zone Mean Air Temperature')]

Notice how the result is a list? If you had looked for surface temperatures instead:

In [76]: dd.find_variable('surface')
Out[76]:
[('TimeStep', 'DPVROOF:1157058.1', 'Surface Outside Face Temperature'),
 ('TimeStep', 'DPVWALL:1157029', 'Surface Outside Face Temperature'),
 ('TimeStep',
  'DPVWINDOW:COMBINED:DPVWALL:1157029:DEFAULTWINDOWCONSTRUCTION',
  'Surface Outside Face Temperature'),
 ('TimeStep', 'DPVWALL:1157028', 'Surface Outside Face Temperature'),
 ('TimeStep', 'DPVROOF:1157058.3', 'Surface Outside Face Temperature'),
 ('TimeStep', 'DPVROOF:1157058.0', 'Surface Outside Face Temperature'),
 ('TimeStep',
  'DPVWINDOW:COMBINED:DPVWALL:1157028:DEFAULTWINDOWCONSTRUCTION',
  'Surface Outside Face Temperature'),
 ('TimeStep',
  'DPVWINDOW:COMBINED:DPVWALL:1157026:DEFAULTWINDOWCONSTRUCTION',
  'Surface Outside Face Temperature'),
 ('TimeStep', 'DPVWALL:1157027', 'Surface Outside Face Temperature'),
 ('TimeStep',
  'DPVWINDOW:COMBINED:DPVWALL:1157027:DEFAULTWINDOWCONSTRUCTION',
  'Surface Outside Face Temperature'),
 ('TimeStep', 'DPVFLOOR:1157042', 'Surface Outside Face Temperature'),
 ('TimeStep', 'DPVWALL:1157026', 'Surface Outside Face Temperature'),
 ('TimeStep', 'DPVROOF:1157058.2', 'Surface Outside Face Temperature')]

you’d have gotten a list of all variables that match ‘surface’ (case-insensitive, substring match). The tuples define the variable you’re looking for: Frequency, key and variable name, since you can have the same variable output for different frequencies and keys!
So the index of the variable we’re looking for (Zone Mean Air Temperature) can be found like this:

In [77]: dd.index['TimeStep', 'DEFAULT_ZONE', 'Zone Mean Air Temperature']
Out[77]: 112

When you parse an .eso file, you get two values back: The DataDictionary and the data itself, which is stored in a simple dictionary mapping the variable index to the timeseries data:

In [82]: dd, data = esoreader.read('RevitToCitySim_fmibeta.eso')

In [84]: data[112]
Out[84]:
[19.9999999999999,
 20.0,
 20.0,
 20.0,
 20.0,
 20.0,
 20.0,

Where does that data come from? From the rest of the .eso file, which looks like this:

1,Zuerich-SMA - - TMY2-66600 WMO#=,  47.38,   8.57,   1.00, 556.00
2,1, 1, 1, 0, 1, 0.00,60.00,Tuesday        
6,-9.733141026918536E-003
99,4.22860281958676
100,2.29752216466107
101,4.62549195332972
102,2.48360878690238
103,4.45346228786434
104,2.40363283464546
105,4.29531948374435
106,2.37522804444541
107,18.
108,4.50377052140968
109,4.62335191215081
110,4.38341749556803
111,4.62165617120029
112,19.9999999999999
278,69070426.0448551
279,2.200249582529068E-006
2,1, 1, 1, 0, 2, 0.00,60.00,Tuesday        
6,-20.0097331410269
99,-3.72067959222121
100,-11.9333570144822
101,-3.2185453285921
102,-11.9539672821419
103,-3.21604754356428
104,-11.6999602208759
105,-3.69978393125912
106,-12.1422252778574
107,18.
108,-0.183133489472591
109,0.134352957894094
110,-0.291113509003108
111,9.541793763769267E-002
112,20.
278,12241499.5530833
279,0.0
2,1, 1, 1, 0, 3, 0.00,60.00,Tuesday        
6,-20.0097331410269
99,-8.25275818278861
100,-10.9050845966823

For all the main variables (id > 5) the format is:

• index
• value

Thus, the data dictionary is necessary to figure out what variables (with what frequency) are being output.
To sum up the tutorial: The code on the pypi page shows you pretty much all you can do and also all you need to do to retrieve a specific timeseries from an .eso file:

• read in the eso file to obtain the data dictionary and the data
• find the key, frequency and variable name you need in the data dictionary (with find_variable) or by guessing from your IDF input
• retrieve the index of that variable
• retrieve the time series data using that index

Introducing RevitPythonShell

RevitPythonShell is a little tool I built to make life in RevitAPI-land a little easier.

When writing plugins for Autodesk Revit Architecture 2010, you have to restart Revit each time you create a new build and manually click your way to the plugins functionality. If you are not so sure about how a given method or property from the API works (or what values to expect when reading it), you will end up with a lot of these edit-compile-run cycles. I don't know how fast your machine is, but starting Revit on mine is not as snappy as I would like. Plus, you can't really experiment, can you?

So, with RevitPythonShell, you can. It embeds IronPython, a .NET port of the python language as a plugin. The main window provides a simple text editor that lets you write a script and execute it. The most useful script is probably this one:

import code
code.interact(None, None, 
        {
            '__name__': '__console__', 
            '__doc__':None, 
            '__revit__': __revit__
        })

This will open up an interactive interpreter loop (known as a REPL), that will let you explore the RevitAPI. And by explore, I mean type a statement, hit enter, see the results, carry on. It doesn't get easier as this!

The above script is actually saved as a "canned command" - an button (in this case named "Interactive") in the toolbar above the main window. Canned commands like these are an ideal place to save scripts you have developed and found useful - a sort of mini-plugin-in-a-plugin architecture.

The default script shown when you start the RevitPythonShell plugin is this one:

# type in a python script to run here...
try:
  # assumes the following file exists: C:\RevitPythonShell\current.py
  import current  
  reload(current)
  
  current.main(__revit__)
except:
  import traceback
  traceback.print_exc()

It imports the module current.py from the search path (normally C:\RevitPythonShell) which contains a script you are currently working on. Clicking the button Execute (or pressing CTRL+RETURN) will execute the contents of the main window, which will in turn import the module and run its main() method.

Loading a script from the file system instead of typing it into the simple editor provided by RevitPythonShell is probably a good idea, since most editors make life a lot easier compared to the TextBox control used here - even Notepad.exe would be preferable!

Note the parameter __revit__ passed into the main method: This is a reference to the Autodesk.Revit.Application API object used by plugins to gain access to Revit.

I will be showing off some of the things you can do with RevitPythonShell in later posts.