Python Vs Ruby Code Examples

Moved from PythonVsRuby:

(See also PythonRubyAttrComparison, PythonRubyInitializer)

In RubyLanguage, every object has a method called "method_missing", which is called every time an undefined method is invoked. You can override method_missing, for super-reflective ease in creating things such as facades. Does PythonLanguage have this feature?

In Python you would override the method __getattr__(self, name). From this method you can either delegate to another method or throw an exception. """The code below is not a good implementation of method_missing, because it does not trap the arguments to the method"""

Do you really need to do that to replicate method_missing?

 class Facade(object):
    def __getattr__(self, what):
            return super(Facade, self).__getattribute__(what)
        except AttributeError?:
            super(Facade, self).__getattribute__("method_missing")(what)

def method_missing(self, what): print "No %s here!"%what

class foo(Facade): def __init__(self): self.x = 1

def new_method_missing(self, what): print "Definitely no %s here!"%what

def test(self): print "1", self.x del self.x print "2", self.x self.method_missing = self.new_method_missing print "3", self.x del self.method_missing print "4", self.x

x = foo() x.test()

I'm writing some code in Python and I have come across a situation where, in Ruby, I would factor out a new control structure. Can anyone tell me how I would refactor this code in Python to meet OnceAndOnlyOnce.

I am using an FTP object to download and upload data. Sometimes there is a long time between downloading and uploading, and the FTP connection times out. So, the FTP object is used by a higher-level object that provides application-specific methods. These methods catch IOError exceptions thrown by the FTP connection, reconnect to the FTP server, do the FTP operation again. They also retry the operation a number of times, delay after multiple failures, etc. etc. So, each method looks something like this:

	def upload( self, data, filename ):
		tmp_filename = self.temp_filename_for( filename )
		attempts = self.attempts
		while 1:
				self.ftp.storbinary( "STOR " + tmp_filename, StringIO(data) )
			except IOError, ex:
				if attempts == 0:
					raise IOError, ex
					if attempts == self.attempts:
						delay( self.retry_delay )
					attempts = attempts - 1

Each method that uses the FTP connection has similar logic, which is repeated in each method. In Ruby, I would apply an ExtractMethod refactoring to extract the common control logic into a new control structure:

	def reconnect_on_timeout
		@attempts.times do
				return yield
			rescue IOError

def upload( data, filename ) reconnect_on_timeout do @ftp.storbinary( "STOR " + tmp_filename, ) end end

def download( filename ) reconnect_on_timeout do blocks = [] @ftp.retrbinary( "RETR " + filename ) { |data| blocks << data } blocks.join("") end end

How would I translate this Ruby code to Python?

How about this (warning: untested code):

	def reconnect_on_timeout(self, operation, parameters): 
		attempts = self.attempts 
		while 1: 
				return apply(operation, parameters)
			except IOError, ex: 
				if attempts == 0: 
					raise IOError, ex 
					delay( self.retry_delay ) 
					attempts = attempts - 1					 

def upload(self, data, filename): self.reconnect_on_timeout(self.uploader, (data, filename))

def uploader(self, data, filename): tmp_filename = self.temp_filename_for(filename) self.ftp.storbinary("STOR " + tmp_filename, StringIO(data))

def download(filename): return self.reconnect_on_timeout(self.downloader, (filename,))

def downloader(self, filename): blocks = [] self.ftp.retrbinary("RETR " + filename, blocks.append) return "".join(blocks)

Another approach might use currying.

First, why does the code above use apply instead of just sending in a function? Even currying is overkill. In part it may because these examples were written on older versions of Python. Today, the Python equivalent is near identical to the Ruby one (and probably could have been much closer even as long ago as five years!).

Second, I'm not 100% clear on why the Original Poster's Python code seems to have extra logic that the Ruby one doesn't. For instance why can the Ruby one use the Ruby equivalent of a for-loop but the Python one uses a while loop, a counter and a "raise" statement (I would guess it is because the original writer wasn't totally comfortable in Python). And the Python one has a guard around the delay that the Ruby one doesn't. I think that the original Python example does more than the shorter, simpler Ruby one. I believe that a transliteration of the Ruby code to Python (with no extra features) would look more like this.

 def reconnect_on_timeout(self, func): 
    attempts = self.attempts 
    for i in attempts: 
            return func()
        except IOError, ex: 

def upload(self, data, filename): # note the closure of data and filename def uploader(): tmp_filename = self.temp_filename_for(filename) self.ftp.storbinary("STOR " + tmp_filename, StringIO(data)) self.reconnect_on_timeout(uploader)

def download(self, filename): # note the closure of filename def downloader(filename): blocks = [] self.ftp.retrbinary("RETR " + filename, blocks.append) return "".join(blocks) return self.reconnect_on_timeout(downloader)

You could also turn the reconnect_on_timeout function into a decorator, and define your functions like this (warning - untested code):

        def reconnect_on_timeout(operation): 
            def function(self, *parameters):
                attempts = self.attempts 
                while 1: 
                                return operation(*parameters)
                        except IOError, ex: 
                                if attempts == 0: 
                                        raise IOError, ex 
                                        delay( self.retry_delay ) 
                                        attempts = attempts - 1
            return function

@reconnect_on_timout def upload(self, data, filename): tmp_filename = self.temp_filename_for(filename) self.ftp.storbinary("STOR " + tmp_filename, StringIO(data))

@reconnect_on_timout def download(self, filename): blocks = [] self.ftp.retrbinary("RETR " + filename, blocks.append) return "".join(blocks)

Then you can call upload & download directly and they will do the right thing.

(untested!) In my opinion, using a higher order function is clearer than inventing a new control structure using "yield".

But yield is the Ruby method for calling block arguments in the current context. In other words, it is a higher order function paradigm. It's just syntactically more conveniant. Ruby, like Python, makes you use special notation to define a function which can be handled anonymously. So I'm not sure where you're taking issue.
How would you write this killer line of Ruby in Python?
 [1,2,3,4,5].sort_by { rand }

Which obviously sorts the array by random values ~ that is shuffles the array! -- Brian

It would be easier to do in Ruby with [1, 2, 3].shuffle.

Like this: random.shuffle([1,2,3,4,5]). I think that's quite clean, although it's inplace, and that might confuse some people.

Oh, you want it without using a builtin function? sorted([5,4,3,2,1], lambda x,y: random.randrange(-1, 1))

Not quite as short, since our random function lives in a namespace and takes different parameters.

or: import random def rand(a, b):
   return random.randint(-1,1)

sorted([5,4,3,2,1], rand) ... though as random.shuffle is the RightThing, neither should be used... ;-)

Be careful with this pattern. Using a random.shuffle library function is the RightThing doing [1,2,3,4,5].sort_by { rand } or it's Python equivalents is usually not the right thing.

random.shuffle is the RightThing for at least three reasons. First, this idiom works because of a subtlety of the (SchwartzianTransform) implementation of Ruby's sort_by. The Python code using the user defined comparison function rand, given above, also depends upon subtle implementation details of the library's sort implementation. Comparisons that aren't transitive, don't give the same answers, or that don't ensure that a == a could under some circumstances cause a sort implementation to run an index off the end of an array or fail in other strange ways. To see this, consider a correctness argument for a sorting algorithm and how the argument depends upon these properties of the comparison operation (e.g.

Second, using sort to do a random shuffle is at least a O(N log N) operation, taking time that grows faster than linearly with the size of the list being shuffled. A proper implementation of random shuffle is only O(N) time.

Third and most interestingly, without care, it is easy to generate shuffles of lists that aren't truly random. This is the case for the Python code fragments. The Ruby idiom does produce a truly random permutation of the list, but only because rand is unlikely to produce two identical values and because Ruby's sort_by uses the Schwartzian Transform internally. See for more information on this subject.

sorted([1, 2, 3, 4, 5], key=lambda x: random.random()) can give list as random as the ruby one.

Ruby Equivalents for Python ListComprehensions

What is the Ruby equivalent of"

 >>> [(x, x*x) for x in [1,2,3,4] if x != 3]
 [(1,1), (2,4), (4,16)]

This accomplishes the same thing, but makes an intermediate list.
 [1,2,3,4].select {|x| x != 3} . map! {|x| [x, x*x]}

Or, without the intermediate array in two passes, which I find more readable:
 [1,2,3,4].map! { |x| [x,x*x] if x != 3 }.compact!
Or the more classic Smalltalk-ish solution in one pass:
 [1,2,3,4].inject( [] ) { |a,x| ( a << [x, x*x] if x != 3 ) || a }
Or perhaps this?
 [1,2,3,4].inject([]) {|a,x| x == 3 ? a : a << [x, x*x]}

I'm not sure I like Python's behavior there though. If you were to rely on your if-statement that way in Ruby, your array would have a nil. I'm pretty sure this makes more sense. Python seems to silently decide that its false value isn't something you keep in that statement. But code like this probably shouldn't see the light of day in a production environment. That goes for the Python example too.

The whole point of a ListComprehension is to select only elements that meet the selection criteria (ie: to build a filtered list). The Ruby selection generates nils, which is not appropriate behavior for a comprehension.

But you're not just filtering. You're both filtering and culling the list. I've seen people proprose methods to do exactly what you're saying on the Array class of Ruby, but usually they are shot down by people who prefer that both verbs show up in such a statement. If all you want is to select off of a list, #reject and #accept handle it.

I find it strange that Python discards those false values.

Python doesn't discard the false values, you're getting confused with LispLanguage. List comprehensions are (more or less) SyntacticSugar for

mylist = [] for val in somecollection:
   if someboolean:

A filter IS something that culls a list:
 >>> filter(lambda x: x%3, range(20))
 [1, 2, 4, 5, 7, 8, 10, 11, 13, 14, 16, 17, 19]

If you want the false values to show up in the list that's trivial with a ListComprehension:
 [(None, n)[bool(n%3)] for n in range(20)]

Or if you want something a little more permanent:
 def null_if_false(n, criterion):
    if criterion(n):
        return n
    return None

>>> [null_if_false(n, lambda x: x%3) for n in x] [None, 1, 2, None, 4, 5, None, 7, 8, None, 10, 11, None, 13, 14, None, 16, 17, None, 19]

That said, I've _never_ever_ needed to use this, and I use ListComprehensions all the time. Note that Python's ListComprehensions build a new list and don't perform in-place modification. The behaviour of Python's filter() and listcomps is hardly original to Python - they're standard constructs of functional languages [though Python's filter() is deprecated in favour of listcomps and gencomps].

AnswerMe: I've used a lot of Python, and this is the first time I've ever seen the idiom:
 (None, n)[bool(criterion(n))]
What the hell is that? Oh, and an aside - apparently, Guido is finally folding on the whole "Python needs Ruby blocks" issue and is working on an implementation of a block in Python that takes an object as it's parameter, and that object is used to determine what to do with the nested code. -- MartinZarate

It's a fairly evil way to emulate the C/Perl/Java idiom "criterion(n) ? n : None" Only "Fairly evil"? ;-)

Actually, it's a bug in the typing system. :) You could use (None,n)[abs(cmp(criterion(n),0))] instead for extra points. :) -- IvanTkatchev Yes, True is 1 and False is 0 when converting to integer in Python.

Python 2.5 has (test ? value_if_true : value_if_false) operator : it is (value_if_true if test else value_if_false). It allows to rewrite this code in a better way :
 >>> [(n if n%3 else None) for n in range(20)]

Sorry about using "filtering and culling", that's what my shop calls it, and I was posting from there. Wrong mindset. As I was saying, in Ruby there are two discrete steps to your operation. One is to select which elements you want to use, and the other is to provide a transform to them. Please note how every example you've given so far reduces the length of the list.

It seems to me like this is a rather trivial difference though. The example below shows one implementation of similar behavior. I'll offer one too, after that one.

I use this idiom in ruby to emulate list comp:
 module Enumerable
   def map_filter(obj=nil)
     each do |x|
       res= yield x
       if res==obj
         acc << res
  alias lc map_filter
 z=  [1,2,3].lc { |x| x*2  if x%2==0 }

I like the way It can optionally handle useful nil return values, and the creative usage of the if statement modifier :)
In the spirit of the example above, here's one that is a little more modern:

 module Enumerable
   def compact_map( comp=nil )
     inject( [] ) do |ary,item|
       t = yield item
       ( ary << t unless t == comp ) || ary

PythonVsRuby has a harder example:

 [(x,y) for x in xrange(3) for y in xrange(3) if x != y] -> [(0, 1), (0, 2), (1, 0), (1, 2), (2, 0), (2, 1)]

I think the Ruby example from PythonVsRuby fits well in contrasting the appraoches the two languages take:

 3.times {|x| 3.times {|y| a << [x,y] if x != y } }


 a=[];for x in xrange(3); for y in xrange(3); a<< [x,y] if x != y end end; a 

class XRange def initialize(n) @n=n end def each() 0.upto(@n-1){|i|yield i} end end def xrange(n) end

This too is lazy, and performance seems about the same (say within a factor of two, slower or faster). It is a bit uglier.

and here is another option (perhaps more Rubyish?):

 xrange(3).inject([]) {|a,x| a + xrange(3).lc {|y| (x == y) ? nil : [x,y] } }

we can define a function to make it cleaner (and use XRange and Enumerable#lc from above):

 module Enumerable
  def collect_flattened(start=[])
     each do |x|
       acc += yield(x)
  alias cf collect_flattened

class XRange include Enumerable end

xrange(3).cf {|x| xrange(3).lc {|y| (x == y) ? nil : [x,y] } }

which seems about as clear from a Ruby standpoint as the list comprehension is from a Python.

One nice feature of the Python version is that it's not cluttered by explicit nesting. The same can be done in Ruby if we abstract out the implicit (lazy) Cartesian product of Enumerables:

 class Product
   include Enumerable
   def initialize *seqs
     @seqs = seqs
   def each &block
     if @seqs.size == 1
       @seqs.first.each &block
       first, *rest = @seqs
       first.each do |x|*rest).each do |xs|
 , *xs)
 end, 0...3).find_all{|x,y| x != y}

This is much cleaner, especially for deeply-nested list comprehensions. If you want Python's combined map/filter semantics, you can use previous posters' compact_map or lc with the product Enumerable.

Or maybe (0..2).to_a.product((0..2).to_a).reject{|a,b| a==b}

Here is a different take on a Ruby equivalent:

  class Array

def comprehend args = [], result=[], &block if empty? then r = yield *args result << r if r else (self[0]||[]).each { |e| self[1..-1].comprehend( args + [e], result, &block) } end result end


This should go into Enumerable, but I put it into Array to make the example more clear:

  [0..2,0..2].comprehend { |x,y| [x,y] if x != y } 

answers the array of pairs where the elements aren't equal:

  [[0, 1], [0, 2], [1, 0], [1, 2], [2, 0], [2, 1]]

The expression

  [0..2,0..2,0..2].comprehend { |x,y,z| [x,y,z] if x != y && y != z } 

answers triples where neither the first two nor the second two pairs are the same:

  [[0, 1, 0],
  [0, 1, 2],
  [0, 2, 0],
  [0, 2, 1],
  [1, 0, 1],
  [1, 0, 2],
  [1, 2, 0],
  [1, 2, 1],
  [2, 0, 1],
  [2, 0, 2],
  [2, 1, 0],
  [2, 1, 2]] 

Anything that responds to "each" can be placed into the array. One can also work with arbitrary arity in the block... the following determines the comprehension for all the combinations where no item is repeated:

  [0..2,0..2,0..2].comprehend { | *x | x if x.uniq.count == x.count  }


  [[0, 1, 2], [0, 2, 1], [1, 0, 2], [1, 2, 0], [2, 0, 1], [2, 1, 0]]

Have fun!


(See also PythonVsRuby, PythonRubyAttrComparison, PythonRubyInitializer)

CategoryProgrammingLanguageComparisons CategoryPython CategoryRuby

View edit of December 4, 2012 or FindPage with title or text search