Htykuut

Is it possible to implement something akin to yield from python in go? That is, I want a producer/consumer paradigm where the consumer can ignore values produced by the producer at any time without manifesting a memory leak.

In python I could have a producer that produces infinite values and the client can consume as many as they wish.

def producer():

  i = 0

  while True:

     yield i

     i += 1



def consumer():

   for i in producer():

     if i == 2:

        return

The common advice for this is for the producer to return a channel that the client uses in a range expression, and the producer can put its values into the channel. This is fine as long as the client reads all the values in the channel, but if the client breaks control from the range loop early the producer channel will be left hanging forever using up memory.

package main



func yield() chan int {

    out := make(chan int)

    go func(){

        out <- 1

        out <- 2

        close(out)

    }()

    return out

}



func test(){

    for _ = range yield() {

        // return

    }

}



func main(){

    for {

        test()

    }

}

If the return in test is uncommented this code will use memory without bound, because the channel returned by yield has a value that is waiting to be read and apparently neither the channel nor the goroutine can be garbage collected safely.

Some possible solutions (neither of which I like) are

1. 
   

   Change out in yield to be a buffered channel with exactly the amount of space as values that will be produced. In that case yield could be

   
   
   

   func yield() chan int {
   out := make(chan int, 2)
   out <- 1
   out <- 2
   close(out)
   return out
   }

   
   

But this is suboptimal because I don't know in general how many items yield will produce. This style is similar to just returning an array.

2. Have the yield method use a timeout for putting values into the channel, and if the timeout is reached then close the channel early. This makes assumptions about the execution time of the client, which seems like a bad idea.