Octave-2-Matlab and back (2)

Like I mentioned earlier here. I finally managed to put together a AST-visitor pattern that
loads the user-script parses it, and gives it to the conversion tree-walker code. This implements
the changes mentioned in the oct2mat package (find it here) by Paul Kienzle .

The email announcing this code and patch was posted

Hello there,

I have a oct2mat like the one Paul Kienzle mentions in his
README file based on the AST walker. Utilizing Octave's
parse tree to do the conversion is very helpful and accurate
to a larger degree. Its still a work in progress (my disclaimer).

Large parts of the code are directly derived from Octave
source written by JWE.

This script is essentially to help deploy code to your colleagues
who want to use it on Matlab systems for whatever reason. Primary
development is still expected to be based on an Octave based
environment.

You can invoke the script as,

octave:1> oct2mat("full/path/my_cool_octave_function.m",1)
octave:2> oct2mat("full/path/my_cool_octave_script.m")

and the code is output to stdout, where from you can use some shell
scripts to redirect the output to another matlab directory.

I have attached the files. Im waiting for JWE to accept some
global export patches to access some parser information. Once that is
done, I plan to add it to the extras branch of octave-forge.

This code works only on the cvs version of Octave, as a external add-on.

Limitations include

1. Can treat only function files or script files not both.
2. Pollutes your command history, as it pipes out to octave_stdout.
3. Cannot handle a script file with function definitions
4. Doesnot write to a separate directory or file, just all output on
stdout. So you need to create a driver program.
5. Not sure if it works correctly for all cases.

Best,
-Muthu


The Code / Tree Walker

I also made patches against existing code from Octave tree (maybe obsolete right now), so
as to help integrate my tree walker into the existing visitor.

Index: pt-pr-code.cc
===================================================================
RCS file: /cvs/octave/src/pt-pr-code.cc,v
retrieving revision 1.54
diff -u -p -r1.54 pt-pr-code.cc
--- pt-pr-code.cc 12 Oct 2007 21:27:33 -0000 1.54
+++ pt-pr-code.cc 10 Jan 2008 05:56:20 -0000
@@ -35,6 +35,240 @@ along with Octave; see the file COPYING.
#include "pr-output.h"
#include "pt-all.h"

+
+static std::string
+matlab_unary_op_as_string (octave_value::unary_op op)
+{
+ std::string retval;
+
+ switch (op)
+ {
+ case octave_value::op_not:
+ retval = "~";
+ break;
+
+ case octave_value::op_uplus:
+ retval = "+";
+ break;
+
+ case octave_value::op_uminus:
+ retval = "-";
+ break;
+
+ case octave_value::op_transpose:
+ retval = ".'";
+ break;
+
+ case octave_value::op_hermitian:
+ retval = "'";
+ break;
+
+ case octave_value::op_incr:
+ retval = "++";
+ break;
+
+ case octave_value::op_decr:
+ retval = "--";
+ break;
+
+ default:
+ retval = "";
+ }
+
+ return retval;
+}
+
+static std::string
+matlab_binary_op_as_string (octave_value::binary_op op)
+{
+ std::string retval;
+
+ switch (op)
+ {
+ case octave_value::op_add:
+ retval = "+";
+ break;
+
+ case octave_value::op_sub:
+ retval = "-";
+ break;
+
+ case octave_value::op_mul:
+ retval = "*";
+ break;
+
+ case octave_value::op_div:
+ retval = "/";
+ break;
+
+ case octave_value::op_pow:
+ retval = "^";
+ break;
+
+ case octave_value::op_ldiv:
+ retval = "\\";
+ break;
+
+ case octave_value::op_lshift:
+ retval = "<<"; + break; + + case octave_value::op_rshift: + retval = ">>";
+ break;
+
+ case octave_value::op_lt:
+ retval = "<"; + break; + + case octave_value::op_le: + retval = "<="; + break; + + case octave_value::op_eq: + retval = "=="; + break; + + case octave_value::op_ge: + retval = ">=";
+ break;
+
+ case octave_value::op_gt:
+ retval = ">";
+ break;
+
+ case octave_value::op_ne:
+ retval = "~=";
+ break;
+
+ case octave_value::op_el_mul:
+ retval = ".*";
+ break;
+
+ case octave_value::op_el_div:
+ retval = "./";
+ break;
+
+ case octave_value::op_el_pow:
+ retval = ".^";
+ break;
+
+ case octave_value::op_el_ldiv:
+ retval = ".\\";
+ break;
+
+ case octave_value::op_el_and:
+ retval = "&";
+ break;
+
+ case octave_value::op_el_or:
+ retval = "|";
+ break;
+
+ case octave_value::op_struct_ref:
+ retval = ".";
+ break;
+
+ default:
+ retval = "";
+ }
+
+ return retval;
+}
+
+static void
+do_oct2mat_oper(std::ostream &os, tree_binary_expression &expr)
+{
+ std::string oprval="";
+
+ // this converts code into whatever form
+ // oct2mat requires.
+ if ( expr.is_boolean_expression() )
+ {
+ const tree_boolean_expression & tbr
+ = dynamic_cast (expr);
+
+ oprval="";
+
+ switch ( tbr.op_type() )
+ {
+ case tree_boolean_expression::bool_and:
+ oprval = "&";
+ break;
+
+ case tree_boolean_expression::bool_or:
+ oprval = "|";
+ break;
+
+ defualt:
+ oprval = "";
+ }
+ }
+ else
+ {
+ oprval=matlab_binary_op_as_string(expr.op_type());
+ }
+
+ os << " " << oprval="">";
+
+ oprval=matlab_unary_op_as_string(expr.op_type());
+ os << " " <<>accept(omat);
+ buf_str = buf.str();
+ os << idx="0;">= 0);
+
+ s << ""; + if (printing_newlines) + { + if (beginning_of_line) + { + s << i =" 0;" beginning_of_line =" false;">accept (*this);

- os << " " << op2 =" expr.rhs">accept (*this);

@@ -237,7 +474,10 @@ tree_print_code::visit_simple_for_comman

indent ();

- os << "endfor"; + if ( convert_to_matlab ) + os << "end"; + else + os << "endfor"; } void @@ -280,7 +520,10 @@ tree_print_code::visit_complex_for_comma indent (); - os << "endfor"; + if ( convert_to_matlab ) + os << "end"; + else + os << "endfor"; } void @@ -401,7 +644,10 @@ tree_print_code::visit_octave_user_funct indent (); - os << "endfunction"; + if ( convert_to_matlab ) + os << "%% endfunction"; + else + os << "endfunction"; newline (); } @@ -414,7 +660,17 @@ tree_print_code::visit_identifier (tree_ print_parens (id, "("); std::string nm = id.name (); - os << (nm.empty () ? std::string ("(empty)") : nm); + + if ( convert_to_matlab ) + { + std::map::iterator iter= identifier_lut.find(nm);
+ if ( iter != identifier_lut.end() )
+ os <<>second ;
+ else
+ os << (nm.empty () ? std::string ("(empty)") : nm); + } + else + os << (nm.empty () ? std::string ("(empty)") : nm); print_parens (id, ")"); } @@ -459,7 +715,10 @@ tree_print_code::visit_if_command (tree_ indent (); - os << "endif"; + if ( convert_to_matlab ) + os << "end"; + else + os << "endif"; } void @@ -693,11 +952,75 @@ tree_print_code::visit_no_op_command (tr void tree_print_code::visit_constant (tree_constant& val) { + std::string buf_str; + std::ostringstream buf; + int idx = 0; + octave_value sval=val.rvalue(); indent (); print_parens (val, "("); - val.print_raw (os, true, print_original_text); + + if ( convert_to_matlab ) + { + if ( sval.is_string() ) + { + buf_str=sval.string_value(); + // see if you need to escape the string stxt. + os << "'"; + for( idx=0; idx < buf_str =" buf.str();" idx="0;">accept (*this);

- os << e =" expr.operand" s =" indent_to_string">accept (*this);
+ // print a commented code.
+ if ( convert_to_matlab )
+ {
+ std::string ind_prfx = indent_to_string( );
+ do_print_commented_code(unwind_protect_code,os,ind_prfx);
+ }
+ else
+ unwind_protect_code->accept (*this);

decrement_indent_level ();
}
@@ -1022,7 +1374,10 @@ tree_print_code::visit_unwind_protect_co

indent ();

- os << "unwind_protect_cleanup"; + if ( convert_to_matlab ) + os << "%%unwind_protect_cleanup"; + else + os << "unwind_protect_cleanup"; newline (); @@ -1032,7 +1387,13 @@ tree_print_code::visit_unwind_protect_co { increment_indent_level (); - cleanup_code->accept (*this);
+ if ( convert_to_matlab )
+ {
+ std::string ind_prfx = indent_to_string( );
+ do_print_commented_code(cleanup_code,os,ind_prfx);
+ }
+ else
+ cleanup_code->accept (*this);

decrement_indent_level ();
}
@@ -1041,7 +1402,10 @@ tree_print_code::visit_unwind_protect_co

indent ();

- os << "end_unwind_protect"; + if ( convert_to_matlab ) + os << "%%end_unwind_protect"; + else + os << "end_unwind_protect"; } void @@ -1075,7 +1439,10 @@ tree_print_code::visit_while_command (tr indent (); - os << "endwhile"; + if ( convert_to_matlab ) + os << "end"; + else + os << "endwhile"; } void @@ -1192,7 +1559,10 @@ tree_print_code::print_comment_elt (cons if (c == '\n') { if (prev_char_was_newline) - os << "##"; + if ( convert_to_matlab ) + os << "%" ; + else + os << "##"; newline (); @@ -1206,7 +1576,11 @@ tree_print_code::print_comment_elt (cons indent (); - os << "##"; + + if ( convert_to_matlab ) + os << "%%"; + else + os << "##"; if (! (isspace (c) || c == '!')) os << " "; Index: pt-pr-code.h =================================================================== RCS file: /cvs/octave/src/pt-pr-code.h,v retrieving revision 1.36 diff -u -p -r1.36 pt-pr-code.h --- pt-pr-code.h 12 Oct 2007 21:27:33 -0000 1.36 +++ pt-pr-code.h 10 Jan 2008 05:56:20 -0000 @@ -26,6 +26,7 @@ along with Octave; see the file COPYING. #include
#include
+#include


I still have bugs in the driver routine, but hey it works!.
-Muthu

Octave-2-Matlab and back

Octave-Lint (3)
A few more additional features that could be copied from existing tools in a similar areas. Immediately we can think of all the warnings that GCC would spit out, when you compile C/C++ code using 'gcc -Wall code.cc'. Specifically, flagging unused variables, data type conversions in known stdlibrary of Octave functions and such. Its time, I could show the code, instead of waxing eloquent about it. Just that I dont have anything to show.

Octave -> Matlab -> Octave
Next in a similar spirit of the AST walker static-checker we could also have a program transformation tool for converting Octave -> Matlab code. I could 'ask' for type inference whenever it is 'confused' by 'ambiguous' nature of the code. This is exciting, in terms of the possibilities it opens up for us. It is going to be a long project, but defnitely rewarding one.
The Octave parser is widely acknowledged on our mailing lists as a superset of Matlab
language. Using the Octave parser, you could in theory convert Matlab code the other-way
round too. Now this is double bonanza, a kind of buy-1 and you get-1 free. Am I excited?

The starting point of the program transformation tool would be from Paul Kienzle's work on
Oct2Mat conversion script. A few things are simple to understand, and require plain substituion. Paul's code does it using an AWK script, and clearly mentions not wanting to Octave's parse tree for doing the conversion. Paul attempts to do it using filters in AWK, and aims for an independent program for doing the conversion. Since our aims are to take advantage of whatever we have (GPL, Octave, Parser) then we can stand on the shoulders of giants instead of 'lets start at the very beginning, Do Re Mi'. Paul's code helps us easily identify the transformation rules to be applied.

The Octave->Matlab transformation rules, applied in Paul's code include

1. gsub("#" , "%"); convert # to %
   
2. gsub("[(][)]",""); convert () to ' ' as Matlab 4.0 and Octave-2.1.50 dont support empty arguments. This is not a problem anymore.
   
3. gsub("gset[^;%]*;",""); a graphics command conversion.
4. gsub("gset[^;]*%","%"); a graphics command conversion.
5. gsub("gset[^;]*$",""); a graphics command conversion.
6. gsub("endfunction",""); Octave has endif, endfor, endfunction etc, so those get converted to end, or in Matlab nothing at all. Infact Matlab complains if you have the end keyword at the end of a function file.
7. gsub("endif","end"); Same as 6.
8. gsub("endwhile","end"); Same as 6.
9. gsub("endfor","end"); Same as 6.
10. gsub("end_try_catch","end"); Same as 6.
11. gsub(/&&/,"\\&"); Matlab didnot have the short circuit logical operators initially, so we had to use the logical &.
12. gsub("SEEK_CUR",0); They also seemingly lacked options for STDIO processing.
13. gsub("SEEK_END",1); Same as 12.
14. gsub("SEEK_SET",-1); Same as 12.
15. gsub("usage","error"); Our usage() and print_usage() are replaced by error() function.
16. gsub("__error_text__","lasterr"); Some Octave specific code
    
17. gsub("unwind_protect_cleanup",""); Again Octave was first here, w.r.t try/catch ...
    
18. gsub("end_unwind_protect",""); ... doing it the LISP way.
    
19. gsub("unwind_protect",""); Same as [17, 18].
    
20. gsub(/\|\|/,"|"); Logical short-circuit OR operator was NOT there in Matlab first. We got here earlier.
    
21. gsub("!","~"); They dont know whats Not !
    
22. gsub("[\\]$","..."); Line continuations are OK for us. Not them!
    
    

These rules are easily loaded as actions for a AST walker, for operators in Octave. We just generate the do_convert_oct_to_mat() on each node. As an example, the node for the operator short-circuit && will have the conversion rule embedded in this routine as follows,

string short_ckt_and::do_convert_oct_to_mat() {
l_str=left_node.do_convert_oct_to_mat();
r_str=right_node.do_convert_oct_to_mat();
str=l_str + " & " + r_str;
return str;
}

So basically it would be a really syntactically correct way of doing things for bulilding transformation tools from the existing code base. As explained earlied since Octave being a superset of Matlab we could also turn the world the other-way round.

Ofcourse the validity of the converter can be verified by doing the rountripping, O-2-M-2-O and running the code to see if we have the regression tests validated. John Eaton (JWE), has a set of regression cases for parser, which is again something we can re-use.

It would be fun to see this happen, and I suspect this is much lower hanging fruit than writing an evaluating AST-walker for building the profiler or a debugger.

Cheers,
Muthu