<commit_before>// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

/*
Package template implements data-driven templates for generating textual output
such as HTML.

Templates are executed by applying them to a data structure. Annotations in the
template refer to elements of the data structure (typically a field of a struct
or a key in a map) to control execution and derive values to be displayed.
Execution of the template walks the structure and sets the cursor, represented
by a period '.' and called "dot", to the value at the current location in the
structure as execution proceeds.

The input text for a template is UTF-8-encoded text in any format.
"Actions"--data evaluations or control structures--are delimited by
"{{" and "}}"; all text outside actions is copied to the output unchanged.
Actions may not span newlines, although comments can.

Once constructed, a template may be executed safely in parallel.

Actions

Here is the list of actions. "Arguments" and "pipelines" are evaluations of
data, defined in detail below.

*/
//	{{/* a comment */}}
//		A comment; discarded. May contain newlines.
//		Comments do not nest.
/*

	{{pipeline}}
		The default textual representation of the value of the pipeline
		is copied to the output.

	{{if pipeline}} T1 {{end}}
		If the value of the pipeline is empty, no output is generated;
		otherwise, T1 is executed.  The empty values are false, 0, any
		nil pointer or interface value, and any array, slice, map, or
		string of length zero.
		Dot is unaffected.

	{{if pipeline}} T1 {{else}} T0 {{end}}
		If the value of the pipeline is empty, T0 is executed;
		otherwise, T1 is executed.  Dot is unaffected.

	{{range pipeline}} T1 {{end}}
		The value of the pipeline must be an array, slice, or map. If
		the value of the pipeline has length zero, nothing is output;
		otherwise, dot is set to the successive elements of the array,
		slice, or map and T1 is executed. If the value is a map and the
		keys are of basic type with a defined order ("comparable"), the
		elements will be visited in sorted key order.

	{{range pipeline}} T1 {{else}} T0 {{end}}
		The value of the pipeline must be an array, slice, or map. If
		the value of the pipeline has length zero, dot is unaffected and
		T0 is executed; otherwise, dot is set to the successive elements
		of the array, slice, or map and T1 is executed.

	{{template "name"}}
		The template with the specified name is executed with nil data.

	{{template "name" pipeline}}
		The template with the specified name is executed with dot set
		to the value of the pipeline.

	{{with pipeline}} T1 {{end}}
		If the value of the pipeline is empty, no output is generated;
		otherwise, dot is set to the value of the pipeline and T1 is
		executed.

	{{with pipeline}} T1 {{else}} T0 {{end}}
		If the value of the pipeline is empty, dot is unaffected and T0
		is executed; otherwise, dot is set to the value of the pipeline
		and T1 is executed.

Arguments

An argument is a simple value, denoted by one of the following.

	- A boolean, string, character, integer, floating-point, imaginary
	  or complex constant in Go syntax. These behave like Go's untyped
	  constants, although raw strings may not span newlines.
	- The character '.' (period):
		.
	  The result is the value of dot.
	- A variable name, which is a (possibly empty) alphanumeric string
	  preceded by a dollar sign, such as
		$piOver2
	  or
		$
	  The result is the value of the variable.
	  Variables are described below.
	- The name of a field of the data, which must be a struct, preceded
	  by a period, such as
		.Field
	  The result is the value of the field. Field invocations may be
	  chained:
	    .Field1.Field2
	  Fields can also be evaluated on variables, including chaining:
	    $x.Field1.Field2
	- The name of a key of the data, which must be a map, preceded
	  by a period, such as
		.Key
	  The result is the map element value indexed by the key.
	  Key invocations may be chained and combined with fields to any
	  depth:
	    .Field1.Key1.Field2.Key2
	  Although the key must be an alphanumeric identifier, unlike with
	  field names they do not need to start with an upper case letter.
	  Keys can also be evaluated on variables, including chaining:
	    $x.key1.key2
	- The name of a niladic method of the data, preceded by a period,
	  such as
		.Method
	  The result is the value of invoking the method with dot as the
	  receiver, dot.Method(). Such a method must have one return value (of
	  any type) or two return values, the second of which is an error.
	  If it has two and the returned error is non-nil, execution terminates
	  and an error is returned to the caller as the value of Execute.
	  Method invocations may be chained and combined with fields and keys
	  to any depth:
	    .Field1.Key1.Method1.Field2.Key2.Method2
	  Methods can also be evaluated on variables, including chaining:
	    $x.Method1.Field
	- The name of a niladic function, such as
		fun
	  The result is the value of invoking the function, fun(). The return
	  types and values behave as in methods. Functions and function
	  names are described below.

Arguments may evaluate to any type; if they are pointers the implementation
automatically indirects to the base type when required.

A pipeline is a possibly chained sequence of "commands". A command is a simple
value (argument) or a function or method call, possibly with multiple arguments:

	Argument
		The result is the value of evaluating the argument.
	.Method [Argument...]
		The method can be alone or the last element of a chain but,
		unlike methods in the middle of a chain, it can take arguments.
		The result is the value of calling the method with the
		arguments:
			dot.Method(Argument1, etc.)
	functionName [Argument...]
		The result is the value of calling the function associated
		with the name:
			function(Argument1, etc.)
		Functions and function names are described below.

Pipelines

A pipeline may be "chained" by separating a sequence of commands with pipeline
characters '|'. In a chained pipeline, the result of the each command is
passed as the last argument of the following command. The output of the final
command in the pipeline is the value of the pipeline.

The output of a command will be either one value or two values, the second of
which has type error. If that second value is present and evaluates to
non-nil, execution terminates and the error is returned to the caller of
Execute.

Variables

A pipeline inside an action may initialize a variable to capture the result.
The initialization has syntax

	$variable := pipeline

where $variable is the name of the variable. An action that declares a
variable produces no output.

If a "range" action initializes a variable, the variable is set to the
successive elements of the iteration.  Also, a "range" may declare two
variables, separated by a comma:

	$index, $element := pipeline

in which case $index and $element are set to the successive values of the
array/slice index or map key and element, respectively.  Note that if there is
only one variable, it is assigned the element; this is opposite to the
convention in Go range clauses.

A variable's scope extends to the "end" action of the control structure ("if",
"with", or "range") in which it is declared, or to the end of the template if
there is no such control structure.  A template invocation does not inherit
variables from the point of its invocation.

When execution begins, $ is set to the data argument passed to Execute, that is,
to the starting value of dot.

Examples

Here are some example one-line templates demonstrating pipelines and variables.
All produce the quoted word "output":

	{{"\"output\""}}
		A string constant.
	{{`"output"`}}
		A raw string constant.
	{{printf "%q" "output"}}
		A function call.
	{{"output" | printf "%q"}}
		A function call whose final argument comes from the previous
		command.
	{{"put" | printf "%s%s" "out" | printf "%q"}}
		A more elaborate call.
	{{"output" | printf "%s" | printf "%q"}}
		A longer chain.
	{{with "output"}}{{printf "%q" .}}{{end}}
		A with action using dot.
	{{with $x := "output" | printf "%q"}}{{$x}}{{end}}
		A with action that creates and uses a variable.
	{{with $x := "output"}}{{printf "%q" $x}}{{end}}
		A with action that uses the variable in another action.
	{{with $x := "output"}}{{$x | printf "%q"}}{{end}}
		The same, but pipelined.

Functions

During execution functions are found in two function maps: first in the
template, then in the global function map. By default, no functions are defined
in the template but the Funcs methods can be used to add them.

Predefined global functions are named as follows.

	and
		Returns the boolean AND of its arguments by returning the
		first empty argument or the last argument, that is,
		"and x y" behaves as "if x then y else x". All the
		arguments are evaluated.
	html
		Returns the escaped HTML equivalent of the textual
		representation of its arguments.
	index
		Returns the result of indexing its first argument by the
		following arguments. Thus "index x 1 2 3" is, in Go syntax,
		x[1][2][3]. Each indexed item must be a map, slice, or array.
	js
		Returns the escaped JavaScript equivalent of the textual
		representation of its arguments.
	len
		Returns the integer length of its argument.
	not
		Returns the boolean negation of its single argument.
	or
		Returns the boolean OR of its arguments by returning the
		first non-empty argument or the last argument, that is,
		"or x y" behaves as "if x then x else y". All the
		arguments are evaluated.
	print
		An alias for fmt.Sprint
	printf
		An alias for fmt.Sprintf
	println
		An alias for fmt.Sprintln
	urlquery
		Returns the escaped value of the textual representation of
		its arguments in a form suitable for embedding in a URL query.

The boolean functions take any zero value to be false and a non-zero value to
be true.

Associated templates

Each template is named by a string specified when it is created. Also, each
template is associated with zero or more other templates that it may invoke by
name; such associations are transitive and form a name space of templates.

A template may use a template invocation to instantiate another associated
template; see the explanation of the "template" action above. The name must be
that of a template associated with the template that contains the invocation.

Nested template definitions

When parsing a template, another template may be defined and associated with the
template being parsed. Template definitions must appear at the top level of the
template, much like global variables in a Go program.

The syntax of such definitions is to surround each template declaration with a
"define" and "end" action.

The define action names the template being created by providing a string
constant. Here is a simple example:

	`{{define "T1"}}ONE{{end}}
	{{define "T2"}}TWO{{end}}
	{{define "T3"}}{{template "T1"}} {{template "T2"}}{{end}}
	{{template "T3"}}`

This defines two templates, T1 and T2, and a third T3 that invokes the other two
when it is executed. Finally it invokes T3. If executed this template will
produce the text

	ONE TWO

By construction, a template may reside in only one association. If it's
necessary to have a template addressable from multiple associations, the
template definition must be parsed multiple times to create distinct *Template
values.

Parse may be called multiple times to assemble the various associated templates;
see the ParseFiles and ParseGlob functions and methods for simple ways to parse
related templates stored in files.

A template may be executed directly or through ExecuteTemplate, which executes
an associated template identified by name. To invoke our example above, we
might write,

	err := tmpl.Execute(os.Stdout, "no data needed")
	if err != nil {
		log.Fatalf("execution failed: %s", err)
	}

or to invoke a particular template explicitly by name,

	err := tmpl.ExecuteTemplate(os.Stdout, "T2", "no data needed")
	if err != nil {
		log.Fatalf("execution failed: %s", err)
	}

*/
package template
<commit_msg>template: refer HTML users to html/template<commit_after>// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

/*
Package template implements data-driven templates for generating textual output.

To generate HTML output, see package html/template, which has the same interface
as this package but automatically secures HTML output against certain attacks.

Templates are executed by applying them to a data structure. Annotations in the
template refer to elements of the data structure (typically a field of a struct
or a key in a map) to control execution and derive values to be displayed.
Execution of the template walks the structure and sets the cursor, represented
by a period '.' and called "dot", to the value at the current location in the
structure as execution proceeds.

The input text for a template is UTF-8-encoded text in any format.
"Actions"--data evaluations or control structures--are delimited by
"{{" and "}}"; all text outside actions is copied to the output unchanged.
Actions may not span newlines, although comments can.

Once constructed, a template may be executed safely in parallel.

Actions

Here is the list of actions. "Arguments" and "pipelines" are evaluations of
data, defined in detail below.

*/
//	{{/* a comment */}}
//		A comment; discarded. May contain newlines.
//		Comments do not nest.
/*

	{{pipeline}}
		The default textual representation of the value of the pipeline
		is copied to the output.

	{{if pipeline}} T1 {{end}}
		If the value of the pipeline is empty, no output is generated;
		otherwise, T1 is executed.  The empty values are false, 0, any
		nil pointer or interface value, and any array, slice, map, or
		string of length zero.
		Dot is unaffected.

	{{if pipeline}} T1 {{else}} T0 {{end}}
		If the value of the pipeline is empty, T0 is executed;
		otherwise, T1 is executed.  Dot is unaffected.

	{{range pipeline}} T1 {{end}}
		The value of the pipeline must be an array, slice, or map. If
		the value of the pipeline has length zero, nothing is output;
		otherwise, dot is set to the successive elements of the array,
		slice, or map and T1 is executed. If the value is a map and the
		keys are of basic type with a defined order ("comparable"), the
		elements will be visited in sorted key order.

	{{range pipeline}} T1 {{else}} T0 {{end}}
		The value of the pipeline must be an array, slice, or map. If
		the value of the pipeline has length zero, dot is unaffected and
		T0 is executed; otherwise, dot is set to the successive elements
		of the array, slice, or map and T1 is executed.

	{{template "name"}}
		The template with the specified name is executed with nil data.

	{{template "name" pipeline}}
		The template with the specified name is executed with dot set
		to the value of the pipeline.

	{{with pipeline}} T1 {{end}}
		If the value of the pipeline is empty, no output is generated;
		otherwise, dot is set to the value of the pipeline and T1 is
		executed.

	{{with pipeline}} T1 {{else}} T0 {{end}}
		If the value of the pipeline is empty, dot is unaffected and T0
		is executed; otherwise, dot is set to the value of the pipeline
		and T1 is executed.

Arguments

An argument is a simple value, denoted by one of the following.

	- A boolean, string, character, integer, floating-point, imaginary
	  or complex constant in Go syntax. These behave like Go's untyped
	  constants, although raw strings may not span newlines.
	- The character '.' (period):
		.
	  The result is the value of dot.
	- A variable name, which is a (possibly empty) alphanumeric string
	  preceded by a dollar sign, such as
		$piOver2
	  or
		$
	  The result is the value of the variable.
	  Variables are described below.
	- The name of a field of the data, which must be a struct, preceded
	  by a period, such as
		.Field
	  The result is the value of the field. Field invocations may be
	  chained:
	    .Field1.Field2
	  Fields can also be evaluated on variables, including chaining:
	    $x.Field1.Field2
	- The name of a key of the data, which must be a map, preceded
	  by a period, such as
		.Key
	  The result is the map element value indexed by the key.
	  Key invocations may be chained and combined with fields to any
	  depth:
	    .Field1.Key1.Field2.Key2
	  Although the key must be an alphanumeric identifier, unlike with
	  field names they do not need to start with an upper case letter.
	  Keys can also be evaluated on variables, including chaining:
	    $x.key1.key2
	- The name of a niladic method of the data, preceded by a period,
	  such as
		.Method
	  The result is the value of invoking the method with dot as the
	  receiver, dot.Method(). Such a method must have one return value (of
	  any type) or two return values, the second of which is an error.
	  If it has two and the returned error is non-nil, execution terminates
	  and an error is returned to the caller as the value of Execute.
	  Method invocations may be chained and combined with fields and keys
	  to any depth:
	    .Field1.Key1.Method1.Field2.Key2.Method2
	  Methods can also be evaluated on variables, including chaining:
	    $x.Method1.Field
	- The name of a niladic function, such as
		fun
	  The result is the value of invoking the function, fun(). The return
	  types and values behave as in methods. Functions and function
	  names are described below.

Arguments may evaluate to any type; if they are pointers the implementation
automatically indirects to the base type when required.

A pipeline is a possibly chained sequence of "commands". A command is a simple
value (argument) or a function or method call, possibly with multiple arguments:

	Argument
		The result is the value of evaluating the argument.
	.Method [Argument...]
		The method can be alone or the last element of a chain but,
		unlike methods in the middle of a chain, it can take arguments.
		The result is the value of calling the method with the
		arguments:
			dot.Method(Argument1, etc.)
	functionName [Argument...]
		The result is the value of calling the function associated
		with the name:
			function(Argument1, etc.)
		Functions and function names are described below.

Pipelines

A pipeline may be "chained" by separating a sequence of commands with pipeline
characters '|'. In a chained pipeline, the result of the each command is
passed as the last argument of the following command. The output of the final
command in the pipeline is the value of the pipeline.

The output of a command will be either one value or two values, the second of
which has type error. If that second value is present and evaluates to
non-nil, execution terminates and the error is returned to the caller of
Execute.

Variables

A pipeline inside an action may initialize a variable to capture the result.
The initialization has syntax

	$variable := pipeline

where $variable is the name of the variable. An action that declares a
variable produces no output.

If a "range" action initializes a variable, the variable is set to the
successive elements of the iteration.  Also, a "range" may declare two
variables, separated by a comma:

	$index, $element := pipeline

in which case $index and $element are set to the successive values of the
array/slice index or map key and element, respectively.  Note that if there is
only one variable, it is assigned the element; this is opposite to the
convention in Go range clauses.

A variable's scope extends to the "end" action of the control structure ("if",
"with", or "range") in which it is declared, or to the end of the template if
there is no such control structure.  A template invocation does not inherit
variables from the point of its invocation.

When execution begins, $ is set to the data argument passed to Execute, that is,
to the starting value of dot.

Examples

Here are some example one-line templates demonstrating pipelines and variables.
All produce the quoted word "output":

	{{"\"output\""}}
		A string constant.
	{{`"output"`}}
		A raw string constant.
	{{printf "%q" "output"}}
		A function call.
	{{"output" | printf "%q"}}
		A function call whose final argument comes from the previous
		command.
	{{"put" | printf "%s%s" "out" | printf "%q"}}
		A more elaborate call.
	{{"output" | printf "%s" | printf "%q"}}
		A longer chain.
	{{with "output"}}{{printf "%q" .}}{{end}}
		A with action using dot.
	{{with $x := "output" | printf "%q"}}{{$x}}{{end}}
		A with action that creates and uses a variable.
	{{with $x := "output"}}{{printf "%q" $x}}{{end}}
		A with action that uses the variable in another action.
	{{with $x := "output"}}{{$x | printf "%q"}}{{end}}
		The same, but pipelined.

Functions

During execution functions are found in two function maps: first in the
template, then in the global function map. By default, no functions are defined
in the template but the Funcs methods can be used to add them.

Predefined global functions are named as follows.

	and
		Returns the boolean AND of its arguments by returning the
		first empty argument or the last argument, that is,
		"and x y" behaves as "if x then y else x". All the
		arguments are evaluated.
	html
		Returns the escaped HTML equivalent of the textual
		representation of its arguments.
	index
		Returns the result of indexing its first argument by the
		following arguments. Thus "index x 1 2 3" is, in Go syntax,
		x[1][2][3]. Each indexed item must be a map, slice, or array.
	js
		Returns the escaped JavaScript equivalent of the textual
		representation of its arguments.
	len
		Returns the integer length of its argument.
	not
		Returns the boolean negation of its single argument.
	or
		Returns the boolean OR of its arguments by returning the
		first non-empty argument or the last argument, that is,
		"or x y" behaves as "if x then x else y". All the
		arguments are evaluated.
	print
		An alias for fmt.Sprint
	printf
		An alias for fmt.Sprintf
	println
		An alias for fmt.Sprintln
	urlquery
		Returns the escaped value of the textual representation of
		its arguments in a form suitable for embedding in a URL query.

The boolean functions take any zero value to be false and a non-zero value to
be true.

Associated templates

Each template is named by a string specified when it is created. Also, each
template is associated with zero or more other templates that it may invoke by
name; such associations are transitive and form a name space of templates.

A template may use a template invocation to instantiate another associated
template; see the explanation of the "template" action above. The name must be
that of a template associated with the template that contains the invocation.

Nested template definitions

When parsing a template, another template may be defined and associated with the
template being parsed. Template definitions must appear at the top level of the
template, much like global variables in a Go program.

The syntax of such definitions is to surround each template declaration with a
"define" and "end" action.

The define action names the template being created by providing a string
constant. Here is a simple example:

	`{{define "T1"}}ONE{{end}}
	{{define "T2"}}TWO{{end}}
	{{define "T3"}}{{template "T1"}} {{template "T2"}}{{end}}
	{{template "T3"}}`

This defines two templates, T1 and T2, and a third T3 that invokes the other two
when it is executed. Finally it invokes T3. If executed this template will
produce the text

	ONE TWO

By construction, a template may reside in only one association. If it's
necessary to have a template addressable from multiple associations, the
template definition must be parsed multiple times to create distinct *Template
values.

Parse may be called multiple times to assemble the various associated templates;
see the ParseFiles and ParseGlob functions and methods for simple ways to parse
related templates stored in files.

A template may be executed directly or through ExecuteTemplate, which executes
an associated template identified by name. To invoke our example above, we
might write,

	err := tmpl.Execute(os.Stdout, "no data needed")
	if err != nil {
		log.Fatalf("execution failed: %s", err)
	}

or to invoke a particular template explicitly by name,

	err := tmpl.ExecuteTemplate(os.Stdout, "T2", "no data needed")
	if err != nil {
		log.Fatalf("execution failed: %s", err)
	}

*/
package template
<|endoftext|>


<commit_before>package fun

import (
	"errors"
	"fmt"
	"testing"
)

func ExampleParallel() {
	Parallel(f1, f2, f3, func(f1res, f2res string, f3res int, err error) {
		fmt.Println(f1res, f2res, f3res)
	})
	// Output:
	// From p1 From p2 123
}

func TestParallel(t *testing.T) {
	Parallel(f1, f2, f3, func(fromP1, fromP2 string, fromP3 int, err error) {
		if err != nil {
			t.Error("Got unexpeted error")
		}
		if fromP1 != "From p1" || fromP2 != "From p2" || fromP3 != 123 {
			t.Fail()
		}
	})
}

func TestParallelErr(t *testing.T) {
	Parallel(f1, fErr, f2, f3, func(fromP1, fromErr, fromP2 string, fromP3 int, err error) {
		if err == nil {
			t.Error("Got nil err")
		} else if err.Error() != "Error from fErr" {
			t.Error("Got incorrect error")
		}
	})
	Parallel(f1, f2, fErr2, f3, func(fromF1, fromF2, fromFErr2 string, fromF3 int, err error) {
		if err == nil {
			t.Error("Got nil err")
		} else if err.Error() != "Error2 from fErr2" {
			t.Error("Got incorrect error")
		}
	})
}

func TestParallelNumInPanic(t *testing.T) {
	defer func() {
		if r := recover(); r == nil {
			t.Error("Did not panic")
		}
	}()
	badParallelFunc := func(unexpectedArgIn int) (res string, err error) { return }
	Parallel(badParallelFunc, func(res string, err error) {})
}

func TestParallelTypeOutPanic(t *testing.T) {
	defer func() {
		r := recover()
		if r != "Parallel function number 0 returns a \"string\" but final function expects a \"int\"" {
			t.Error("Did not panic with the expected message")
		}
	}()
	badParallelOutputFun := func() (res string, err error) { return }
	Parallel(badParallelOutputFun, func(res int, err error) {})
}

func TestParallelFinalErrArgPanic(t *testing.T) {
	defer func() {
		if r := recover(); r != "Parallel final function's last argument type should be error but is int" {
			t.Error("Did not panic with the expected message")
		}
	}()

	Parallel(f1, f2, func(res1, res2 string, err int) {})
}

func TestParallelFinalFuncReturnsError(t *testing.T) {
	var err error
	err = Parallel(f1, f2, func(res1, res2 string, err error) {})
	assert(t, err == nil)
	err = Parallel(f1, fErr, func(res1, res2 string, err error) {})
	assert(t, err != nil)
	err = Parallel(f1, f2, func(res1, res2 string, err error) error { return nil })
	assert(t, err == nil)
	err = Parallel(f1, fErr, func(res1, res2 string, err error) error { return errors.New("A new error") })
	assert(t, err != nil)
	assert(t, err.Error() == "A new error")
	err = Parallel(f1, f2, func(res1, res2 string, err error) error { return nil })
	assert(t, err == nil)
}

func assert(t *testing.T, shouldBeTrue bool) {
	if shouldBeTrue {
		return
	}
	t.Fail()
}

func fErr2() (res string, err error) {
	err = errors.New("Error2 from fErr2")
	return
}

func fErr() (res string, err error) {
	err = errors.New("Error from fErr")
	return
}
func f3() (int, error) {
	return 123, nil
}
func f1() (string, error) {
	return "From p1", nil
}
func f2() (string, error) {
	return "From p2", nil
}
<commit_msg>Easy-to-comment-out panic for test debugging<commit_after>package fun

import (
	"errors"
	"fmt"
	"testing"
)

func ExampleParallel() {
	Parallel(f1, f2, f3, func(f1res, f2res string, f3res int, err error) {
		fmt.Println(f1res, f2res, f3res)
	})
	// Output:
	// From p1 From p2 123
}

func TestParallel(t *testing.T) {
	Parallel(f1, f2, f3, func(fromP1, fromP2 string, fromP3 int, err error) {
		if err != nil {
			t.Error("Got unexpeted error")
		}
		if fromP1 != "From p1" || fromP2 != "From p2" || fromP3 != 123 {
			t.Fail()
		}
	})
}

func TestParallelErr(t *testing.T) {
	Parallel(f1, fErr, f2, f3, func(fromP1, fromErr, fromP2 string, fromP3 int, err error) {
		if err == nil {
			t.Error("Got nil err")
		} else if err.Error() != "Error from fErr" {
			t.Error("Got incorrect error")
		}
	})
	Parallel(f1, f2, fErr2, f3, func(fromF1, fromF2, fromFErr2 string, fromF3 int, err error) {
		if err == nil {
			t.Error("Got nil err")
		} else if err.Error() != "Error2 from fErr2" {
			t.Error("Got incorrect error")
		}
	})
}

func TestParallelNumInPanic(t *testing.T) {
	defer func() {
		if r := recover(); r == nil {
			t.Error("Did not panic")
		}
	}()
	badParallelFunc := func(unexpectedArgIn int) (res string, err error) { return }
	Parallel(badParallelFunc, func(res string, err error) {})
}

func TestParallelTypeOutPanic(t *testing.T) {
	defer func() {
		r := recover()
		if r != "Parallel function number 0 returns a \"string\" but final function expects a \"int\"" {
			t.Error("Did not panic with the expected message")
		}
	}()
	badParallelOutputFun := func() (res string, err error) { return }
	Parallel(badParallelOutputFun, func(res int, err error) {})
}

func TestParallelFinalErrArgPanic(t *testing.T) {
	defer func() {
		if r := recover(); r != "Parallel final function's last argument type should be error but is int" {
			t.Error("Did not panic with the expected message")
		}
	}()

	Parallel(f1, f2, func(res1, res2 string, err int) {})
}

func TestParallelFinalFuncReturnsError(t *testing.T) {
	var err error
	err = Parallel(f1, f2, func(res1, res2 string, err error) {})
	assert(t, err == nil)
	err = Parallel(f1, fErr, func(res1, res2 string, err error) {})
	assert(t, err != nil)
	err = Parallel(f1, f2, func(res1, res2 string, err error) error { return nil })
	assert(t, err == nil)
	err = Parallel(f1, fErr, func(res1, res2 string, err error) error { return errors.New("A new error") })
	assert(t, err != nil)
	assert(t, err.Error() == "A new error")
	err = Parallel(f1, f2, func(res1, res2 string, err error) error { return nil })
	assert(t, err == nil)
}

func assert(t *testing.T, shouldBeTrue bool) {
	if shouldBeTrue {
		return
	}
	t.Error("assert failed")
	// panic("assert failed")
}

func fErr2() (res string, err error) {
	err = errors.New("Error2 from fErr2")
	return
}

func fErr() (res string, err error) {
	err = errors.New("Error from fErr")
	return
}
func f3() (int, error) {
	return 123, nil
}
func f1() (string, error) {
	return "From p1", nil
}
func f2() (string, error) {
	return "From p2", nil
}
<|endoftext|>


<commit_before>package test

import (
	"github.com/stephenalexbrowne/zoom"
	"github.com/stephenalexbrowne/zoom/redis"
	"github.com/stephenalexbrowne/zoom/support"
	"github.com/stephenalexbrowne/zoom/util"
	"testing"
)

func TestSaveOneToOne(t *testing.T) {
	support.SetUp()
	defer support.TearDown()

	// create and save a new color
	c := &support.Color{R: 25, G: 152, B: 166}
	zoom.Save(c)

	// create and save a new artist, assigning favoriteColor to above
	a := &support.Artist{Name: "Alex", FavoriteColor: c}
	if err := zoom.Save(a); err != nil {
		t.Error(err)
	}

	// get a connection
	conn := zoom.GetConn()
	defer conn.Close()

	// invoke redis driver to check if the value was set appropriately
	colorKey := "artist:" + a.Id + ":FavoriteColor"
	id, err := redis.String(conn.Do("GET", colorKey))
	if err != nil {
		t.Error(err)
	}
	if id != c.Id {
		t.Errorf("color id for artist was not set correctly.\nExpected: %s\nGot: %s\n", c.Id, id)
	}
}

func TestFindOneToOne(t *testing.T) {
	support.SetUp()
	defer support.TearDown()

	// create and save a new color
	c := &support.Color{R: 25, G: 152, B: 166}
	zoom.Save(c)

	// create and save a new artist, assigning favoriteColor to above
	a := &support.Artist{Name: "Alex", FavoriteColor: c}
	zoom.Save(a)

	// find the saved person
	aCopy := &support.Artist{}
	if _, err := zoom.ScanById(aCopy, a.Id).Exec(); err != nil {
		t.Error(err)
	}

	// make sure favorite color is the same
	if aCopy.FavoriteColor == nil {
		t.Error("relation was not persisted. aCopy.FavoriteColor was nil")
	}
	if a.FavoriteColor.Id != aCopy.FavoriteColor.Id {
		t.Errorf("Id of favorite color was incorrect.\nExpected: %s\nGot: %s\n", a.FavoriteColor.Id, aCopy.FavoriteColor.Id)
	}
}

func TestSaveOneToMany(t *testing.T) {
	support.SetUp()
	defer support.TearDown()

	// create and save a new petOwner
	owners, err := support.CreatePetOwners(1)
	if err != nil {
		t.Error(err)
	}
	o := owners[0]

	// create and save some pets
	pets, err := support.CreatePets(3)
	if err != nil {
		t.Error(err)
	}

	// assign the pets to the owner
	o.Pets = pets
	if err := zoom.Save(o); err != nil {
		t.Error(err)
	}

	// get a connection
	conn := zoom.GetConn()
	defer conn.Close()

	// invoke redis driver to check if the value was set appropriately
	petsKey := "petOwner:" + o.Id + ":Pets"
	gotIds, err := redis.Strings(conn.Do("SMEMBERS", petsKey))
	if err != nil {
		t.Error(err)
	}

	// compare expected ids to got ids
	expectedIds := make([]string, 0)
	for _, pet := range o.Pets {
		if pet.Id == "" {
			t.Errorf("pet id was empty for %+v\n", pet)
		}
		expectedIds = append(expectedIds, pet.Id)
	}
	equal, msg := util.CompareAsStringSet(expectedIds, gotIds)
	if !equal {
		t.Errorf("pet ids were not correct.\n%s\n", msg)
	}
}

func TestFindOneToMany(t *testing.T) {
	support.SetUp()
	defer support.TearDown()

	// create and save a new petOwner
	owners, _ := support.CreatePetOwners(1)
	o := owners[0]

	// create and save some pets
	pets, _ := support.CreatePets(3)

	// assign the pets to the owner
	o.Pets = pets
	zoom.Save(o)

	// get a copy of the owner from the database
	oCopy := &support.PetOwner{}
	if _, err := zoom.ScanById(oCopy, o.Id).Exec(); err != nil {
		t.Error(err)
	}

	// compare expected ids to got ids
	expectedIds := make([]string, 0)
	for _, pet := range o.Pets {
		if pet.Id == "" {
			t.Errorf("pet id was empty for %+v\n", pet)
		}
		expectedIds = append(expectedIds, pet.Id)
	}
	gotIds := make([]string, 0)
	for _, pet := range oCopy.Pets {
		if pet.Id == "" {
			t.Errorf("pet id was empty for %+v\n", pet)
		}
		gotIds = append(gotIds, pet.Id)
	}
	equal, msg := util.CompareAsStringSet(expectedIds, gotIds)
	if !equal {
		t.Errorf("pet ids were not correct.\n%s\n", msg)
	}
}
<commit_msg>Add test for excluded relations<commit_after>package test

import (
	"github.com/stephenalexbrowne/zoom"
	"github.com/stephenalexbrowne/zoom/redis"
	"github.com/stephenalexbrowne/zoom/support"
	"github.com/stephenalexbrowne/zoom/util"
	"testing"
)

func TestSaveOneToOne(t *testing.T) {
	support.SetUp()
	defer support.TearDown()

	// create and save a new color
	c := &support.Color{R: 25, G: 152, B: 166}
	zoom.Save(c)

	// create and save a new artist, assigning favoriteColor to above
	a := &support.Artist{Name: "Alex", FavoriteColor: c}
	if err := zoom.Save(a); err != nil {
		t.Error(err)
	}

	// get a connection
	conn := zoom.GetConn()
	defer conn.Close()

	// invoke redis driver to check if the value was set appropriately
	colorKey := "artist:" + a.Id + ":FavoriteColor"
	id, err := redis.String(conn.Do("GET", colorKey))
	if err != nil {
		t.Error(err)
	}
	if id != c.Id {
		t.Errorf("color id for artist was not set correctly.\nExpected: %s\nGot: %s\n", c.Id, id)
	}
}

func TestFindOneToOne(t *testing.T) {
	support.SetUp()
	defer support.TearDown()

	// create and save a new color
	c := &support.Color{R: 25, G: 152, B: 166}
	zoom.Save(c)

	// create and save a new artist, assigning favoriteColor to above
	a := &support.Artist{Name: "Alex", FavoriteColor: c}
	zoom.Save(a)

	// find the saved person
	aCopy := &support.Artist{}
	if _, err := zoom.ScanById(aCopy, a.Id).Exec(); err != nil {
		t.Error(err)
	}

	// make sure favorite color is the same
	if aCopy.FavoriteColor == nil {
		t.Error("relation was not persisted. aCopy.FavoriteColor was nil")
	}
	if a.FavoriteColor.Id != aCopy.FavoriteColor.Id {
		t.Errorf("Id of favorite color was incorrect.\nExpected: %s\nGot: %s\n", a.FavoriteColor.Id, aCopy.FavoriteColor.Id)
	}
}

func TestSaveOneToMany(t *testing.T) {
	support.SetUp()
	defer support.TearDown()

	// create and save a new petOwner
	owners, err := support.CreatePetOwners(1)
	if err != nil {
		t.Error(err)
	}
	o := owners[0]

	// create and save some pets
	pets, err := support.CreatePets(3)
	if err != nil {
		t.Error(err)
	}

	// assign the pets to the owner
	o.Pets = pets
	if err := zoom.Save(o); err != nil {
		t.Error(err)
	}

	// get a connection
	conn := zoom.GetConn()
	defer conn.Close()

	// invoke redis driver to check if the value was set appropriately
	petsKey := "petOwner:" + o.Id + ":Pets"
	gotIds, err := redis.Strings(conn.Do("SMEMBERS", petsKey))
	if err != nil {
		t.Error(err)
	}

	// compare expected ids to got ids
	expectedIds := make([]string, 0)
	for _, pet := range o.Pets {
		if pet.Id == "" {
			t.Errorf("pet id was empty for %+v\n", pet)
		}
		expectedIds = append(expectedIds, pet.Id)
	}
	equal, msg := util.CompareAsStringSet(expectedIds, gotIds)
	if !equal {
		t.Errorf("pet ids were not correct.\n%s\n", msg)
	}
}

func TestFindOneToMany(t *testing.T) {
	support.SetUp()
	defer support.TearDown()

	// create and save a new petOwner
	owners, _ := support.CreatePetOwners(1)
	o := owners[0]

	// create and save some pets
	pets, _ := support.CreatePets(3)

	// assign the pets to the owner
	o.Pets = pets
	zoom.Save(o)

	// get a copy of the owner from the database
	oCopy := &support.PetOwner{}
	if _, err := zoom.ScanById(oCopy, o.Id).Exec(); err != nil {
		t.Error(err)
	}

	// compare expected ids to got ids
	expectedIds := make([]string, 0)
	for _, pet := range o.Pets {
		if pet.Id == "" {
			t.Errorf("pet id was empty for %+v\n", pet)
		}
		expectedIds = append(expectedIds, pet.Id)
	}
	gotIds := make([]string, 0)
	for _, pet := range oCopy.Pets {
		if pet.Id == "" {
			t.Errorf("pet id was empty for %+v\n", pet)
		}
		gotIds = append(gotIds, pet.Id)
	}
	equal, msg := util.CompareAsStringSet(expectedIds, gotIds)
	if !equal {
		t.Errorf("pet ids were not correct.\n%s\n", msg)
	}
}

func TestFindOneToOneExclude(t *testing.T) {
	support.SetUp()
	defer support.TearDown()

	// create and save a new color
	c := &support.Color{R: 25, G: 152, B: 166}
	zoom.Save(c)

	// create and save a new artist, assigning favoriteColor to above
	a := &support.Artist{Name: "Alex", FavoriteColor: c}
	zoom.Save(a)

	// find the saved person
	aCopy := &support.Artist{}
	if _, err := zoom.ScanById(aCopy, a.Id).Exclude("FavoriteColor").Exec(); err != nil {
		t.Error(err)
	}

	// make sure favorite color is nil
	if aCopy.FavoriteColor != nil {
		t.Errorf("excluded relation was not empty. aCopy.FavoriteColor was: ", aCopy.FavoriteColor)
	}

	// make sure Name was still set
	if aCopy.Name != "Alex" {
		t.Errorf("artist Name was incorrect.\nExpected: %s\nWas: %s\n", "Alex", aCopy.Name)
	}
}

func TestFindOneToManyExclude(t *testing.T) {
	support.SetUp()
	defer support.TearDown()

	// create and save a new petOwner
	owners, _ := support.CreatePetOwners(1)
	o := owners[0]

	// create and save some pets
	pets, _ := support.CreatePets(3)

	// assign the pets to the owner
	o.Pets = pets
	zoom.Save(o)

	// get a copy of the owner from the database
	oCopy := &support.PetOwner{}
	if _, err := zoom.ScanById(oCopy, o.Id).Exclude("Pets").Exec(); err != nil {
		t.Error(err)
	}

	// make sure pets is nil
	if oCopy.Pets != nil {
		t.Errorf("excluded relation was not empty. oCopy.Pets was: ", oCopy.Pets)
	}

	// make sure name was still set
	if oCopy.Name != o.Name {
		t.Errorf("artist Name was incorrect.\nExpected: %s\nWas: %s\n", o.Name, oCopy.Name)
	}
}
<|endoftext|>


<commit_before>/*
Copyright 2016 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package app

import (
	"context"
	"fmt"
	"net/http"
	"os"
	"time"

	"github.com/spf13/cobra"

	"k8s.io/apimachinery/pkg/util/sets"
	"k8s.io/apimachinery/pkg/util/uuid"
	"k8s.io/apimachinery/pkg/util/wait"
	"k8s.io/apiserver/pkg/server"
	"k8s.io/apiserver/pkg/server/healthz"
	apiserverflag "k8s.io/apiserver/pkg/util/flag"
	"k8s.io/apiserver/pkg/util/globalflag"
	"k8s.io/client-go/tools/leaderelection"
	"k8s.io/client-go/tools/leaderelection/resourcelock"
	cloudprovider "k8s.io/cloud-provider"
	"k8s.io/klog"
	cloudcontrollerconfig "k8s.io/kubernetes/cmd/cloud-controller-manager/app/config"
	"k8s.io/kubernetes/cmd/cloud-controller-manager/app/options"
	genericcontrollermanager "k8s.io/kubernetes/cmd/controller-manager/app"
	cmoptions "k8s.io/kubernetes/cmd/controller-manager/app/options"
	"k8s.io/kubernetes/pkg/util/configz"
	utilflag "k8s.io/kubernetes/pkg/util/flag"
	"k8s.io/kubernetes/pkg/version"
	"k8s.io/kubernetes/pkg/version/verflag"
)

const (
	// ControllerStartJitter is the jitter value used when starting controller managers.
	ControllerStartJitter = 1.0
	// ConfigzName is the name used for register cloud-controller manager /configz, same with GroupName.
	ConfigzName = "cloudcontrollermanager.config.k8s.io"
)

// NewCloudControllerManagerCommand creates a *cobra.Command object with default parameters
func NewCloudControllerManagerCommand() *cobra.Command {
	s, err := options.NewCloudControllerManagerOptions()
	if err != nil {
		klog.Fatalf("unable to initialize command options: %v", err)
	}

	cmd := &cobra.Command{
		Use: "cloud-controller-manager",
		Long: `The Cloud controller manager is a daemon that embeds
the cloud specific control loops shipped with Kubernetes.`,
		Run: func(cmd *cobra.Command, args []string) {
			verflag.PrintAndExitIfRequested()
			utilflag.PrintFlags(cmd.Flags())

			c, err := s.Config(KnownControllers(), ControllersDisabledByDefault.List())
			if err != nil {
				fmt.Fprintf(os.Stderr, "%v\n", err)
				os.Exit(1)
			}

			if err := Run(c.Complete(), wait.NeverStop); err != nil {
				fmt.Fprintf(os.Stderr, "%v\n", err)
				os.Exit(1)
			}

		},
	}

	fs := cmd.Flags()
	namedFlagSets := s.Flags(KnownControllers(), ControllersDisabledByDefault.List())
	verflag.AddFlags(namedFlagSets.FlagSet("global"))
	globalflag.AddGlobalFlags(namedFlagSets.FlagSet("global"), cmd.Name())
	cmoptions.AddCustomGlobalFlags(namedFlagSets.FlagSet("generic"))
	for _, f := range namedFlagSets.FlagSets {
		fs.AddFlagSet(f)
	}
	usageFmt := "Usage:\n  %s\n"
	cols, _, _ := apiserverflag.TerminalSize(cmd.OutOrStdout())
	cmd.SetUsageFunc(func(cmd *cobra.Command) error {
		fmt.Fprintf(cmd.OutOrStderr(), usageFmt, cmd.UseLine())
		apiserverflag.PrintSections(cmd.OutOrStderr(), namedFlagSets, cols)
		return nil
	})
	cmd.SetHelpFunc(func(cmd *cobra.Command, args []string) {
		fmt.Fprintf(cmd.OutOrStdout(), "%s\n\n"+usageFmt, cmd.Long, cmd.UseLine())
		apiserverflag.PrintSections(cmd.OutOrStdout(), namedFlagSets, cols)
	})

	return cmd
}

// Run runs the ExternalCMServer.  This should never exit.
func Run(c *cloudcontrollerconfig.CompletedConfig, stopCh <-chan struct{}) error {
	// To help debugging, immediately log version
	klog.Infof("Version: %+v", version.Get())

	cloud, err := cloudprovider.InitCloudProvider(c.ComponentConfig.KubeCloudShared.CloudProvider.Name, c.ComponentConfig.KubeCloudShared.CloudProvider.CloudConfigFile)
	if err != nil {
		klog.Fatalf("Cloud provider could not be initialized: %v", err)
	}
	if cloud == nil {
		klog.Fatalf("cloud provider is nil")
	}

	if cloud.HasClusterID() == false {
		if c.ComponentConfig.KubeCloudShared.AllowUntaggedCloud == true {
			klog.Warning("detected a cluster without a ClusterID.  A ClusterID will be required in the future.  Please tag your cluster to avoid any future issues")
		} else {
			klog.Fatalf("no ClusterID found.  A ClusterID is required for the cloud provider to function properly.  This check can be bypassed by setting the allow-untagged-cloud option")
		}
	}

	// setup /configz endpoint
	if cz, err := configz.New(ConfigzName); err == nil {
		cz.Set(c.ComponentConfig)
	} else {
		klog.Errorf("unable to register configz: %c", err)
	}

	// Setup any healthz checks we will want to use.
	var checks []healthz.HealthzChecker
	var electionChecker *leaderelection.HealthzAdaptor
	if c.ComponentConfig.Generic.LeaderElection.LeaderElect {
		electionChecker = leaderelection.NewLeaderHealthzAdaptor(time.Second * 20)
		checks = append(checks, electionChecker)
	}

	// Start the controller manager HTTP server
	if c.SecureServing != nil {
		unsecuredMux := genericcontrollermanager.NewBaseHandler(&c.ComponentConfig.Generic.Debugging, checks...)
		handler := genericcontrollermanager.BuildHandlerChain(unsecuredMux, &c.Authorization, &c.Authentication)
		if err := c.SecureServing.Serve(handler, 0, stopCh); err != nil {
			return err
		}
	}
	if c.InsecureServing != nil {
		unsecuredMux := genericcontrollermanager.NewBaseHandler(&c.ComponentConfig.Generic.Debugging, checks...)
		insecureSuperuserAuthn := server.AuthenticationInfo{Authenticator: &server.InsecureSuperuser{}}
		handler := genericcontrollermanager.BuildHandlerChain(unsecuredMux, nil, &insecureSuperuserAuthn)
		if err := c.InsecureServing.Serve(handler, 0, stopCh); err != nil {
			return err
		}
	}

	run := func(ctx context.Context) {
		if err := startControllers(c, ctx.Done(), cloud, newControllerInitializers()); err != nil {
			klog.Fatalf("error running controllers: %v", err)
		}
	}

	if !c.ComponentConfig.Generic.LeaderElection.LeaderElect {
		run(context.TODO())
		panic("unreachable")
	}

	// Identity used to distinguish between multiple cloud controller manager instances
	id, err := os.Hostname()
	if err != nil {
		return err
	}
	// add a uniquifier so that two processes on the same host don't accidentally both become active
	id = id + "_" + string(uuid.NewUUID())

	// Lock required for leader election
	rl, err := resourcelock.New(c.ComponentConfig.Generic.LeaderElection.ResourceLock,
		"kube-system",
		"cloud-controller-manager",
		c.LeaderElectionClient.CoreV1(),
		resourcelock.ResourceLockConfig{
			Identity:      id,
			EventRecorder: c.EventRecorder,
		})
	if err != nil {
		klog.Fatalf("error creating lock: %v", err)
	}

	// Try and become the leader and start cloud controller manager loops
	leaderelection.RunOrDie(context.TODO(), leaderelection.LeaderElectionConfig{
		Lock:          rl,
		LeaseDuration: c.ComponentConfig.Generic.LeaderElection.LeaseDuration.Duration,
		RenewDeadline: c.ComponentConfig.Generic.LeaderElection.RenewDeadline.Duration,
		RetryPeriod:   c.ComponentConfig.Generic.LeaderElection.RetryPeriod.Duration,
		Callbacks: leaderelection.LeaderCallbacks{
			OnStartedLeading: run,
			OnStoppedLeading: func() {
				klog.Fatalf("leaderelection lost")
			},
		},
		WatchDog: electionChecker,
		Name:     "cloud-controller-manager",
	})
	panic("unreachable")
}

// startControllers starts the cloud specific controller loops.
func startControllers(c *cloudcontrollerconfig.CompletedConfig, stopCh <-chan struct{}, cloud cloudprovider.Interface, controllers map[string]initFunc) error {
	if cloud != nil {
		// Initialize the cloud provider with a reference to the clientBuilder
		cloud.Initialize(c.ClientBuilder, stopCh)
	}

	for controllerName, initFn := range controllers {
		if !genericcontrollermanager.IsControllerEnabled(controllerName, ControllersDisabledByDefault, c.ComponentConfig.Generic.Controllers) {
			klog.Warningf("%q is disabled", controllerName)
			continue
		}

		klog.V(1).Infof("Starting %q", controllerName)
		_, started, err := initFn(c, cloud, stopCh)
		if err != nil {
			klog.Errorf("Error starting %q", controllerName)
			return err
		}
		if !started {
			klog.Warningf("Skipping %q", controllerName)
			continue
		}
		klog.Infof("Started %q", controllerName)

		time.Sleep(wait.Jitter(c.ComponentConfig.Generic.ControllerStartInterval.Duration, ControllerStartJitter))
	}

	// If apiserver is not running we should wait for some time and fail only then. This is particularly
	// important when we start apiserver and controller manager at the same time.
	if err := genericcontrollermanager.WaitForAPIServer(c.VersionedClient, 10*time.Second); err != nil {
		klog.Fatalf("Failed to wait for apiserver being healthy: %v", err)
	}

	c.SharedInformers.Start(stopCh)

	select {}
}

// initFunc is used to launch a particular controller.  It may run additional "should I activate checks".
// Any error returned will cause the controller process to `Fatal`
// The bool indicates whether the controller was enabled.
type initFunc func(ctx *cloudcontrollerconfig.CompletedConfig, cloud cloudprovider.Interface, stop <-chan struct{}) (debuggingHandler http.Handler, enabled bool, err error)

// KnownControllers indicate the default controller we are known.
func KnownControllers() []string {
	ret := sets.StringKeySet(newControllerInitializers())
	return ret.List()
}

// ControllersDisabledByDefault is the controller disabled default when starting cloud-controller managers.
var ControllersDisabledByDefault = sets.NewString()

// newControllerInitializers is a private map of named controller groups (you can start more than one in an init func)
// paired to their initFunc.  This allows for structured downstream composition and subdivision.
func newControllerInitializers() map[string]initFunc {
	controllers := map[string]initFunc{}
	controllers["cloud-node"] = startCloudNodeController
	controllers["cloud-node-lifecycle"] = startCloudNodeLifecycleController
	controllers["persistentvolume-binder"] = startPersistentVolumeLabelController
	controllers["service"] = startServiceController
	controllers["route"] = startRouteController
	return controllers
}
<commit_msg>Set the informer on the user cloud object<commit_after>/*
Copyright 2016 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package app

import (
	"context"
	"fmt"
	"net/http"
	"os"
	"time"

	"github.com/spf13/cobra"

	"k8s.io/apimachinery/pkg/util/sets"
	"k8s.io/apimachinery/pkg/util/uuid"
	"k8s.io/apimachinery/pkg/util/wait"
	"k8s.io/apiserver/pkg/server"
	"k8s.io/apiserver/pkg/server/healthz"
	apiserverflag "k8s.io/apiserver/pkg/util/flag"
	"k8s.io/apiserver/pkg/util/globalflag"
	"k8s.io/client-go/tools/leaderelection"
	"k8s.io/client-go/tools/leaderelection/resourcelock"
	cloudprovider "k8s.io/cloud-provider"
	"k8s.io/klog"
	cloudcontrollerconfig "k8s.io/kubernetes/cmd/cloud-controller-manager/app/config"
	"k8s.io/kubernetes/cmd/cloud-controller-manager/app/options"
	genericcontrollermanager "k8s.io/kubernetes/cmd/controller-manager/app"
	cmoptions "k8s.io/kubernetes/cmd/controller-manager/app/options"
	"k8s.io/kubernetes/pkg/util/configz"
	utilflag "k8s.io/kubernetes/pkg/util/flag"
	"k8s.io/kubernetes/pkg/version"
	"k8s.io/kubernetes/pkg/version/verflag"
)

const (
	// ControllerStartJitter is the jitter value used when starting controller managers.
	ControllerStartJitter = 1.0
	// ConfigzName is the name used for register cloud-controller manager /configz, same with GroupName.
	ConfigzName = "cloudcontrollermanager.config.k8s.io"
)

// NewCloudControllerManagerCommand creates a *cobra.Command object with default parameters
func NewCloudControllerManagerCommand() *cobra.Command {
	s, err := options.NewCloudControllerManagerOptions()
	if err != nil {
		klog.Fatalf("unable to initialize command options: %v", err)
	}

	cmd := &cobra.Command{
		Use: "cloud-controller-manager",
		Long: `The Cloud controller manager is a daemon that embeds
the cloud specific control loops shipped with Kubernetes.`,
		Run: func(cmd *cobra.Command, args []string) {
			verflag.PrintAndExitIfRequested()
			utilflag.PrintFlags(cmd.Flags())

			c, err := s.Config(KnownControllers(), ControllersDisabledByDefault.List())
			if err != nil {
				fmt.Fprintf(os.Stderr, "%v\n", err)
				os.Exit(1)
			}

			if err := Run(c.Complete(), wait.NeverStop); err != nil {
				fmt.Fprintf(os.Stderr, "%v\n", err)
				os.Exit(1)
			}

		},
	}

	fs := cmd.Flags()
	namedFlagSets := s.Flags(KnownControllers(), ControllersDisabledByDefault.List())
	verflag.AddFlags(namedFlagSets.FlagSet("global"))
	globalflag.AddGlobalFlags(namedFlagSets.FlagSet("global"), cmd.Name())
	cmoptions.AddCustomGlobalFlags(namedFlagSets.FlagSet("generic"))
	for _, f := range namedFlagSets.FlagSets {
		fs.AddFlagSet(f)
	}
	usageFmt := "Usage:\n  %s\n"
	cols, _, _ := apiserverflag.TerminalSize(cmd.OutOrStdout())
	cmd.SetUsageFunc(func(cmd *cobra.Command) error {
		fmt.Fprintf(cmd.OutOrStderr(), usageFmt, cmd.UseLine())
		apiserverflag.PrintSections(cmd.OutOrStderr(), namedFlagSets, cols)
		return nil
	})
	cmd.SetHelpFunc(func(cmd *cobra.Command, args []string) {
		fmt.Fprintf(cmd.OutOrStdout(), "%s\n\n"+usageFmt, cmd.Long, cmd.UseLine())
		apiserverflag.PrintSections(cmd.OutOrStdout(), namedFlagSets, cols)
	})

	return cmd
}

// Run runs the ExternalCMServer.  This should never exit.
func Run(c *cloudcontrollerconfig.CompletedConfig, stopCh <-chan struct{}) error {
	// To help debugging, immediately log version
	klog.Infof("Version: %+v", version.Get())

	cloud, err := cloudprovider.InitCloudProvider(c.ComponentConfig.KubeCloudShared.CloudProvider.Name, c.ComponentConfig.KubeCloudShared.CloudProvider.CloudConfigFile)
	if err != nil {
		klog.Fatalf("Cloud provider could not be initialized: %v", err)
	}
	if cloud == nil {
		klog.Fatalf("cloud provider is nil")
	}

	if cloud.HasClusterID() == false {
		if c.ComponentConfig.KubeCloudShared.AllowUntaggedCloud == true {
			klog.Warning("detected a cluster without a ClusterID.  A ClusterID will be required in the future.  Please tag your cluster to avoid any future issues")
		} else {
			klog.Fatalf("no ClusterID found.  A ClusterID is required for the cloud provider to function properly.  This check can be bypassed by setting the allow-untagged-cloud option")
		}
	}

	// setup /configz endpoint
	if cz, err := configz.New(ConfigzName); err == nil {
		cz.Set(c.ComponentConfig)
	} else {
		klog.Errorf("unable to register configz: %c", err)
	}

	// Setup any healthz checks we will want to use.
	var checks []healthz.HealthzChecker
	var electionChecker *leaderelection.HealthzAdaptor
	if c.ComponentConfig.Generic.LeaderElection.LeaderElect {
		electionChecker = leaderelection.NewLeaderHealthzAdaptor(time.Second * 20)
		checks = append(checks, electionChecker)
	}

	// Start the controller manager HTTP server
	if c.SecureServing != nil {
		unsecuredMux := genericcontrollermanager.NewBaseHandler(&c.ComponentConfig.Generic.Debugging, checks...)
		handler := genericcontrollermanager.BuildHandlerChain(unsecuredMux, &c.Authorization, &c.Authentication)
		if err := c.SecureServing.Serve(handler, 0, stopCh); err != nil {
			return err
		}
	}
	if c.InsecureServing != nil {
		unsecuredMux := genericcontrollermanager.NewBaseHandler(&c.ComponentConfig.Generic.Debugging, checks...)
		insecureSuperuserAuthn := server.AuthenticationInfo{Authenticator: &server.InsecureSuperuser{}}
		handler := genericcontrollermanager.BuildHandlerChain(unsecuredMux, nil, &insecureSuperuserAuthn)
		if err := c.InsecureServing.Serve(handler, 0, stopCh); err != nil {
			return err
		}
	}

	run := func(ctx context.Context) {
		if err := startControllers(c, ctx.Done(), cloud, newControllerInitializers()); err != nil {
			klog.Fatalf("error running controllers: %v", err)
		}
	}

	if !c.ComponentConfig.Generic.LeaderElection.LeaderElect {
		run(context.TODO())
		panic("unreachable")
	}

	// Identity used to distinguish between multiple cloud controller manager instances
	id, err := os.Hostname()
	if err != nil {
		return err
	}
	// add a uniquifier so that two processes on the same host don't accidentally both become active
	id = id + "_" + string(uuid.NewUUID())

	// Lock required for leader election
	rl, err := resourcelock.New(c.ComponentConfig.Generic.LeaderElection.ResourceLock,
		"kube-system",
		"cloud-controller-manager",
		c.LeaderElectionClient.CoreV1(),
		resourcelock.ResourceLockConfig{
			Identity:      id,
			EventRecorder: c.EventRecorder,
		})
	if err != nil {
		klog.Fatalf("error creating lock: %v", err)
	}

	// Try and become the leader and start cloud controller manager loops
	leaderelection.RunOrDie(context.TODO(), leaderelection.LeaderElectionConfig{
		Lock:          rl,
		LeaseDuration: c.ComponentConfig.Generic.LeaderElection.LeaseDuration.Duration,
		RenewDeadline: c.ComponentConfig.Generic.LeaderElection.RenewDeadline.Duration,
		RetryPeriod:   c.ComponentConfig.Generic.LeaderElection.RetryPeriod.Duration,
		Callbacks: leaderelection.LeaderCallbacks{
			OnStartedLeading: run,
			OnStoppedLeading: func() {
				klog.Fatalf("leaderelection lost")
			},
		},
		WatchDog: electionChecker,
		Name:     "cloud-controller-manager",
	})
	panic("unreachable")
}

// startControllers starts the cloud specific controller loops.
func startControllers(c *cloudcontrollerconfig.CompletedConfig, stopCh <-chan struct{}, cloud cloudprovider.Interface, controllers map[string]initFunc) error {
	if cloud != nil {
		// Initialize the cloud provider with a reference to the clientBuilder
		cloud.Initialize(c.ClientBuilder, stopCh)
		// Set the informer on the user cloud object
		if informerUserCloud, ok := cloud.(cloudprovider.InformerUser); ok {
			informerUserCloud.SetInformers(c.SharedInformers)
		}
	}

	for controllerName, initFn := range controllers {
		if !genericcontrollermanager.IsControllerEnabled(controllerName, ControllersDisabledByDefault, c.ComponentConfig.Generic.Controllers) {
			klog.Warningf("%q is disabled", controllerName)
			continue
		}

		klog.V(1).Infof("Starting %q", controllerName)
		_, started, err := initFn(c, cloud, stopCh)
		if err != nil {
			klog.Errorf("Error starting %q", controllerName)
			return err
		}
		if !started {
			klog.Warningf("Skipping %q", controllerName)
			continue
		}
		klog.Infof("Started %q", controllerName)

		time.Sleep(wait.Jitter(c.ComponentConfig.Generic.ControllerStartInterval.Duration, ControllerStartJitter))
	}

	// If apiserver is not running we should wait for some time and fail only then. This is particularly
	// important when we start apiserver and controller manager at the same time.
	if err := genericcontrollermanager.WaitForAPIServer(c.VersionedClient, 10*time.Second); err != nil {
		klog.Fatalf("Failed to wait for apiserver being healthy: %v", err)
	}

	c.SharedInformers.Start(stopCh)

	select {}
}

// initFunc is used to launch a particular controller.  It may run additional "should I activate checks".
// Any error returned will cause the controller process to `Fatal`
// The bool indicates whether the controller was enabled.
type initFunc func(ctx *cloudcontrollerconfig.CompletedConfig, cloud cloudprovider.Interface, stop <-chan struct{}) (debuggingHandler http.Handler, enabled bool, err error)

// KnownControllers indicate the default controller we are known.
func KnownControllers() []string {
	ret := sets.StringKeySet(newControllerInitializers())
	return ret.List()
}

// ControllersDisabledByDefault is the controller disabled default when starting cloud-controller managers.
var ControllersDisabledByDefault = sets.NewString()

// newControllerInitializers is a private map of named controller groups (you can start more than one in an init func)
// paired to their initFunc.  This allows for structured downstream composition and subdivision.
func newControllerInitializers() map[string]initFunc {
	controllers := map[string]initFunc{}
	controllers["cloud-node"] = startCloudNodeController
	controllers["cloud-node-lifecycle"] = startCloudNodeLifecycleController
	controllers["persistentvolume-binder"] = startPersistentVolumeLabelController
	controllers["service"] = startServiceController
	controllers["route"] = startRouteController
	return controllers
}
<|endoftext|>


<commit_before>// Copyright 2019 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0

package e2e

import "testing"

func TestListSetters(t *testing.T) {
	tests := []test{
		{
			name: "set",
			args: []string{"cfg", "list-setters", "."},
			files: map[string]string{
				"deployment.yaml": `
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3  # {"$openapi":"replicas"}
`,
				"Krmfile": `
apiVersion: config.k8s.io/v1alpha1
kind: Krmfile
openAPI:
  definitions:
    io.k8s.cli.setters.replicas:
      x-k8s-cli:
        setter:
          name: replicas
          value: "3"
`,
			},
			expectedStdOut: `
./
    NAME     VALUE   SET BY   DESCRIPTION   COUNT   REQUIRED
  replicas   3                              1       No
`,
		},
	}
	runTests(t, tests)
}
<commit_msg>Fix list_setters test.<commit_after>// Copyright 2019 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0

package e2e

import "testing"

func TestListSetters(t *testing.T) {
	tests := []test{
		{
			name: "set",
			args: []string{"cfg", "list-setters", "."},
			files: map[string]string{
				"deployment.yaml": `
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3  # {"$openapi":"replicas"}
`,
				"Krmfile": `
apiVersion: config.k8s.io/v1alpha1
kind: Krmfile
openAPI:
  definitions:
    io.k8s.cli.setters.replicas:
      x-k8s-cli:
        setter:
          name: replicas
          value: "3"
`,
			},
			expectedStdOut: `
./
    NAME     VALUE   IS SET   SET BY   DESCRIPTION   COUNT   REQUIRED
  replicas   3       No                              1       No
`,
		},
	}
	runTests(t, tests)
}
<|endoftext|>


<commit_before>package distribution

import (
	"fmt"
	"os"
	"strings"

	"github.com/Sirupsen/logrus"
	"github.com/docker/distribution/reference"
	"github.com/docker/docker/api/types"
	"github.com/docker/docker/daemon/events"
	"github.com/docker/docker/distribution/metadata"
	"github.com/docker/docker/distribution/xfer"
	"github.com/docker/docker/image"
	"github.com/docker/docker/pkg/progress"
	"github.com/docker/docker/registry"
	"github.com/docker/docker/tag"
	"golang.org/x/net/context"
)

// ImagePullConfig stores pull configuration.
type ImagePullConfig struct {
	// MetaHeaders stores HTTP headers with metadata about the image
	// (DockerHeaders with prefix X-Meta- in the request).
	MetaHeaders map[string][]string
	// AuthConfig holds authentication credentials for authenticating with
	// the registry.
	AuthConfig *types.AuthConfig
	// ProgressOutput is the interface for showing the status of the pull
	// operation.
	ProgressOutput progress.Output
	// RegistryService is the registry service to use for TLS configuration
	// and endpoint lookup.
	RegistryService *registry.Service
	// EventsService is the events service to use for logging.
	EventsService *events.Events
	// MetadataStore is the storage backend for distribution-specific
	// metadata.
	MetadataStore metadata.Store
	// ImageStore manages images.
	ImageStore image.Store
	// TagStore manages tags.
	TagStore tag.Store
	// DownloadManager manages concurrent pulls.
	DownloadManager *xfer.LayerDownloadManager
}

// Puller is an interface that abstracts pulling for different API versions.
type Puller interface {
	// Pull tries to pull the image referenced by `tag`
	// Pull returns an error if any, as well as a boolean that determines whether to retry Pull on the next configured endpoint.
	//
	Pull(ctx context.Context, ref reference.Named) (fallback bool, err error)
}

// newPuller returns a Puller interface that will pull from either a v1 or v2
// registry. The endpoint argument contains a Version field that determines
// whether a v1 or v2 puller will be created. The other parameters are passed
// through to the underlying puller implementation for use during the actual
// pull operation.
func newPuller(endpoint registry.APIEndpoint, repoInfo *registry.RepositoryInfo, imagePullConfig *ImagePullConfig) (Puller, error) {
	switch endpoint.Version {
	case registry.APIVersion2:
		return &v2Puller{
			blobSumService: metadata.NewBlobSumService(imagePullConfig.MetadataStore),
			endpoint:       endpoint,
			config:         imagePullConfig,
			repoInfo:       repoInfo,
		}, nil
	case registry.APIVersion1:
		return &v1Puller{
			v1IDService: metadata.NewV1IDService(imagePullConfig.MetadataStore),
			endpoint:    endpoint,
			config:      imagePullConfig,
			repoInfo:    repoInfo,
		}, nil
	}
	return nil, fmt.Errorf("unknown version %d for registry %s", endpoint.Version, endpoint.URL)
}

// Pull initiates a pull operation. image is the repository name to pull, and
// tag may be either empty, or indicate a specific tag to pull.
func Pull(ctx context.Context, ref reference.Named, imagePullConfig *ImagePullConfig) error {
	// Resolve the Repository name from fqn to RepositoryInfo
	repoInfo, err := imagePullConfig.RegistryService.ResolveRepository(ref)
	if err != nil {
		return err
	}

	// makes sure name is not empty or `scratch`
	if err := validateRepoName(repoInfo.LocalName.Name()); err != nil {
		return err
	}

	endpoints, err := imagePullConfig.RegistryService.LookupPullEndpoints(repoInfo.CanonicalName)
	if err != nil {
		return err
	}

	logName := registry.NormalizeLocalReference(ref)

	var (
		// use a slice to append the error strings and return a joined string to caller
		errors []string

		// discardNoSupportErrors is used to track whether an endpoint encountered an error of type registry.ErrNoSupport
		// By default it is false, which means that if a ErrNoSupport error is encountered, it will be saved in errors.
		// As soon as another kind of error is encountered, discardNoSupportErrors is set to true, avoiding the saving of
		// any subsequent ErrNoSupport errors in errors.
		// It's needed for pull-by-digest on v1 endpoints: if there are only v1 endpoints configured, the error should be
		// returned and displayed, but if there was a v2 endpoint which supports pull-by-digest, then the last relevant
		// error is the ones from v2 endpoints not v1.
		discardNoSupportErrors bool
	)
	for _, endpoint := range endpoints {
		logrus.Debugf("Trying to pull %s from %s %s", repoInfo.LocalName, endpoint.URL, endpoint.Version)

		puller, err := newPuller(endpoint, repoInfo, imagePullConfig)
		if err != nil {
			errors = append(errors, err.Error())
			continue
		}
		if fallback, err := puller.Pull(ctx, ref); err != nil {
			// Was this pull cancelled? If so, don't try to fall
			// back.
			select {
			case <-ctx.Done():
				fallback = false
			default:
			}
			if fallback {
				if _, ok := err.(registry.ErrNoSupport); !ok {
					// Because we found an error that's not ErrNoSupport, discard all subsequent ErrNoSupport errors.
					discardNoSupportErrors = true
					// append subsequent errors
					errors = append(errors, err.Error())
				} else if !discardNoSupportErrors {
					// Save the ErrNoSupport error, because it's either the first error or all encountered errors
					// were also ErrNoSupport errors.
					// append subsequent errors
					errors = append(errors, err.Error())
				}
				continue
			}
			errors = append(errors, err.Error())
			logrus.Debugf("Not continuing with error: %v", fmt.Errorf(strings.Join(errors, "\n")))
			if len(errors) > 0 {
				return fmt.Errorf(strings.Join(errors, "\n"))
			}
		}

		imagePullConfig.EventsService.Log("pull", logName.String(), "")
		return nil
	}

	if len(errors) == 0 {
		return fmt.Errorf("no endpoints found for %s", ref.String())
	}

	if len(errors) > 0 {
		return fmt.Errorf(strings.Join(errors, "\n"))
	}
	return nil
}

// writeStatus writes a status message to out. If layersDownloaded is true, the
// status message indicates that a newer image was downloaded. Otherwise, it
// indicates that the image is up to date. requestedTag is the tag the message
// will refer to.
func writeStatus(requestedTag string, out progress.Output, layersDownloaded bool) {
	if layersDownloaded {
		progress.Message(out, "", "Status: Downloaded newer image for "+requestedTag)
	} else {
		progress.Message(out, "", "Status: Image is up to date for "+requestedTag)
	}
}

// validateRepoName validates the name of a repository.
func validateRepoName(name string) error {
	if name == "" {
		return fmt.Errorf("Repository name can't be empty")
	}
	if name == "scratch" {
		return fmt.Errorf("'scratch' is a reserved name")
	}
	return nil
}

// tmpFileClose creates a closer function for a temporary file that closes the file
// and also deletes it.
func tmpFileCloser(tmpFile *os.File) func() error {
	return func() error {
		tmpFile.Close()
		if err := os.RemoveAll(tmpFile.Name()); err != nil {
			logrus.Errorf("Failed to remove temp file: %s", tmpFile.Name())
		}

		return nil
	}
}
<commit_msg>The logName is confused here, it's a localName actually.<commit_after>package distribution

import (
	"fmt"
	"os"
	"strings"

	"github.com/Sirupsen/logrus"
	"github.com/docker/distribution/reference"
	"github.com/docker/docker/api/types"
	"github.com/docker/docker/daemon/events"
	"github.com/docker/docker/distribution/metadata"
	"github.com/docker/docker/distribution/xfer"
	"github.com/docker/docker/image"
	"github.com/docker/docker/pkg/progress"
	"github.com/docker/docker/registry"
	"github.com/docker/docker/tag"
	"golang.org/x/net/context"
)

// ImagePullConfig stores pull configuration.
type ImagePullConfig struct {
	// MetaHeaders stores HTTP headers with metadata about the image
	// (DockerHeaders with prefix X-Meta- in the request).
	MetaHeaders map[string][]string
	// AuthConfig holds authentication credentials for authenticating with
	// the registry.
	AuthConfig *types.AuthConfig
	// ProgressOutput is the interface for showing the status of the pull
	// operation.
	ProgressOutput progress.Output
	// RegistryService is the registry service to use for TLS configuration
	// and endpoint lookup.
	RegistryService *registry.Service
	// EventsService is the events service to use for logging.
	EventsService *events.Events
	// MetadataStore is the storage backend for distribution-specific
	// metadata.
	MetadataStore metadata.Store
	// ImageStore manages images.
	ImageStore image.Store
	// TagStore manages tags.
	TagStore tag.Store
	// DownloadManager manages concurrent pulls.
	DownloadManager *xfer.LayerDownloadManager
}

// Puller is an interface that abstracts pulling for different API versions.
type Puller interface {
	// Pull tries to pull the image referenced by `tag`
	// Pull returns an error if any, as well as a boolean that determines whether to retry Pull on the next configured endpoint.
	//
	Pull(ctx context.Context, ref reference.Named) (fallback bool, err error)
}

// newPuller returns a Puller interface that will pull from either a v1 or v2
// registry. The endpoint argument contains a Version field that determines
// whether a v1 or v2 puller will be created. The other parameters are passed
// through to the underlying puller implementation for use during the actual
// pull operation.
func newPuller(endpoint registry.APIEndpoint, repoInfo *registry.RepositoryInfo, imagePullConfig *ImagePullConfig) (Puller, error) {
	switch endpoint.Version {
	case registry.APIVersion2:
		return &v2Puller{
			blobSumService: metadata.NewBlobSumService(imagePullConfig.MetadataStore),
			endpoint:       endpoint,
			config:         imagePullConfig,
			repoInfo:       repoInfo,
		}, nil
	case registry.APIVersion1:
		return &v1Puller{
			v1IDService: metadata.NewV1IDService(imagePullConfig.MetadataStore),
			endpoint:    endpoint,
			config:      imagePullConfig,
			repoInfo:    repoInfo,
		}, nil
	}
	return nil, fmt.Errorf("unknown version %d for registry %s", endpoint.Version, endpoint.URL)
}

// Pull initiates a pull operation. image is the repository name to pull, and
// tag may be either empty, or indicate a specific tag to pull.
func Pull(ctx context.Context, ref reference.Named, imagePullConfig *ImagePullConfig) error {
	// Resolve the Repository name from fqn to RepositoryInfo
	repoInfo, err := imagePullConfig.RegistryService.ResolveRepository(ref)
	if err != nil {
		return err
	}

	// makes sure name is not empty or `scratch`
	if err := validateRepoName(repoInfo.LocalName.Name()); err != nil {
		return err
	}

	endpoints, err := imagePullConfig.RegistryService.LookupPullEndpoints(repoInfo.CanonicalName)
	if err != nil {
		return err
	}

	localName := registry.NormalizeLocalReference(ref)

	var (
		// use a slice to append the error strings and return a joined string to caller
		errors []string

		// discardNoSupportErrors is used to track whether an endpoint encountered an error of type registry.ErrNoSupport
		// By default it is false, which means that if a ErrNoSupport error is encountered, it will be saved in errors.
		// As soon as another kind of error is encountered, discardNoSupportErrors is set to true, avoiding the saving of
		// any subsequent ErrNoSupport errors in errors.
		// It's needed for pull-by-digest on v1 endpoints: if there are only v1 endpoints configured, the error should be
		// returned and displayed, but if there was a v2 endpoint which supports pull-by-digest, then the last relevant
		// error is the ones from v2 endpoints not v1.
		discardNoSupportErrors bool
	)
	for _, endpoint := range endpoints {
		logrus.Debugf("Trying to pull %s from %s %s", repoInfo.LocalName, endpoint.URL, endpoint.Version)

		puller, err := newPuller(endpoint, repoInfo, imagePullConfig)
		if err != nil {
			errors = append(errors, err.Error())
			continue
		}
		if fallback, err := puller.Pull(ctx, ref); err != nil {
			// Was this pull cancelled? If so, don't try to fall
			// back.
			select {
			case <-ctx.Done():
				fallback = false
			default:
			}
			if fallback {
				if _, ok := err.(registry.ErrNoSupport); !ok {
					// Because we found an error that's not ErrNoSupport, discard all subsequent ErrNoSupport errors.
					discardNoSupportErrors = true
					// append subsequent errors
					errors = append(errors, err.Error())
				} else if !discardNoSupportErrors {
					// Save the ErrNoSupport error, because it's either the first error or all encountered errors
					// were also ErrNoSupport errors.
					// append subsequent errors
					errors = append(errors, err.Error())
				}
				continue
			}
			errors = append(errors, err.Error())
			logrus.Debugf("Not continuing with error: %v", fmt.Errorf(strings.Join(errors, "\n")))
			if len(errors) > 0 {
				return fmt.Errorf(strings.Join(errors, "\n"))
			}
		}

		imagePullConfig.EventsService.Log("pull", localName.String(), "")
		return nil
	}

	if len(errors) == 0 {
		return fmt.Errorf("no endpoints found for %s", ref.String())
	}

	if len(errors) > 0 {
		return fmt.Errorf(strings.Join(errors, "\n"))
	}
	return nil
}

// writeStatus writes a status message to out. If layersDownloaded is true, the
// status message indicates that a newer image was downloaded. Otherwise, it
// indicates that the image is up to date. requestedTag is the tag the message
// will refer to.
func writeStatus(requestedTag string, out progress.Output, layersDownloaded bool) {
	if layersDownloaded {
		progress.Message(out, "", "Status: Downloaded newer image for "+requestedTag)
	} else {
		progress.Message(out, "", "Status: Image is up to date for "+requestedTag)
	}
}

// validateRepoName validates the name of a repository.
func validateRepoName(name string) error {
	if name == "" {
		return fmt.Errorf("Repository name can't be empty")
	}
	if name == "scratch" {
		return fmt.Errorf("'scratch' is a reserved name")
	}
	return nil
}

// tmpFileClose creates a closer function for a temporary file that closes the file
// and also deletes it.
func tmpFileCloser(tmpFile *os.File) func() error {
	return func() error {
		tmpFile.Close()
		if err := os.RemoveAll(tmpFile.Name()); err != nil {
			logrus.Errorf("Failed to remove temp file: %s", tmpFile.Name())
		}

		return nil
	}
}
<|endoftext|>


<commit_before>package services_state

import (
	"encoding/json"
	"fmt"
	"testing"
	"time"

	. "github.com/smartystreets/goconvey/convey"
	"github.com/newrelic/bosun/service"
)

var hostname = "shakespeare"
var anotherHostname = "chaucer"

func Test_NewServer(t *testing.T) {

	Convey("Invoking NewServer()", t, func() {
		Convey("Returns a server with the correct name", func() {
			server := NewServer(hostname)
			So(server.Name, ShouldEqual, hostname)
		})

		Convey("Initializes the map", func() {
			server := NewServer(hostname)
			So(server.Services, ShouldNotBeNil)
		})

		Convey("Initializes the time", func() {
			server := NewServer(hostname)
			So(server.LastUpdated, ShouldBeTheSameTimeAs, time.Unix(0, 0))
		})
	})
}

func Test_NewServicesState(t *testing.T) {
	Convey("Invoking NewServicesState()", t, func() {

		Convey("Initializes the Servers map", func() {
			state := NewServicesState()
			So(state.Servers, ShouldNotBeNil)
		})

	})
}

func Test_ServicesStateWithData(t *testing.T) {

	Convey("When working with data", t, func() {
		state := NewServicesState()
		state.Servers[hostname] = NewServer(hostname)

		baseTime := time.Now().UTC()

		svc := service.Service{
			ID: "deadbeef123",
			Name: "radical_service",
			Image: "101deadbeef",
			Created: baseTime,
			Hostname: anotherHostname,
			Updated: baseTime,
			Status: service.ALIVE,
		}

		Convey("Encode() generates JSON that we can Decode()", func() {
			decoded, err := Decode(state.Encode())

			So(err, ShouldBeNil)
			So(decoded.Servers[hostname].Name, ShouldEqual, hostname)
			So(len(decoded.Servers), ShouldEqual, 1)
		})

		Convey("Decode() returns an error when handed junk", func() {
			result, err := Decode([]byte("asdf"))

			So(result.Servers, ShouldBeEmpty)
			So(err, ShouldNotBeNil)
		})

		Convey("HasServer() is true when a server exists", func() {
			So(state.HasServer(hostname), ShouldBeTrue)
		})

		Convey("HasServer() is false when a server is missing", func() {
			So(state.HasServer("junk"), ShouldBeFalse)
		})

		Convey("AddServiceEntry()", func() {
			Convey("Merges in a new service", func() {
				So(state.HasServer(anotherHostname), ShouldBeFalse)

				state.AddServiceEntry(svc)

				So(state.HasServer(anotherHostname), ShouldBeTrue)
				So(state.Servers[anotherHostname].Services[svc.ID], ShouldNotBeNil)
			})

			Convey("Doesn't merge a stale service", func() {
				state.AddServiceEntry(svc)

				staleService := service.Service{
					ID: "deadbeef123",
					Name: "stale_service",
					Image: "stale",
					Created: baseTime,
					Hostname: anotherHostname,
					Updated: baseTime.Add(0 - 1 * time.Minute),
					Status: service.ALIVE,
				}

				state.AddServiceEntry(staleService)

				So(state.HasServer(anotherHostname), ShouldBeTrue)
				So(state.Servers[anotherHostname].Services[svc.ID].Updated,
					ShouldBeTheSameTimeAs, baseTime)
				So(state.Servers[anotherHostname].Services[svc.ID].Image,
					ShouldEqual, "101deadbeef")
			})

			Convey("Updates the LastUpdated time for the server", func() {
				newDate := svc.Updated.AddDate(0, 0, 5)
				svc.Updated = newDate
				state.AddServiceEntry(svc)

				So(state.Servers[anotherHostname].LastUpdated, ShouldBeTheSameTimeAs, newDate)
			})
		})

		Convey("Format() pretty-prints the state even without a Memberlist", func() {
			formatted := state.Format(nil)

			So(formatted, ShouldNotBeNil)
		})

		Reset(func() {
			state = NewServicesState()
			state.Servers[hostname] = NewServer(hostname)
		})
	})
}

func Test_Broadcasts(t *testing.T) {

	Convey("When Broadcasting services", t, func() {
		state := NewServicesState()
		state.Servers[hostname] = NewServer(hostname)
		broadcasts := make(chan [][]byte)
		quit       := make(chan bool)
		svcId1     := "deadbeef123"
		svcId2     := "deadbeef101"
		baseTime   := time.Now().UTC()

		service1 := service.Service{ ID: svcId1, Hostname: hostname, Updated: baseTime }
		service2 := service.Service{ ID: svcId2, Hostname: hostname, Updated: baseTime }
		services := []service.Service{ service1, service2 }

		containerFn := func() []service.Service {
			return services
		}

		state.HostnameFn = func() (string, error) { return hostname, nil }

		Convey("New services are serialized into the channel", func() {
			go func() { quit <- true }()
			go state.BroadcastServices(broadcasts, containerFn, quit)

			json1, _ := json.Marshal(service1)
			json2, _ := json.Marshal(service2)

			readBroadcasts := <-broadcasts
			So(len(readBroadcasts), ShouldEqual, 2)
			So(string(readBroadcasts[0]), ShouldEqual, string(json1))
			So(string(readBroadcasts[1]), ShouldEqual, string(json2))
		})

		Convey("All of the services are added to state", func() {
			go func() { quit <- true }()
			go state.BroadcastServices(broadcasts, containerFn, quit)
			<-broadcasts // Block until we get a result

			So(state.Servers[hostname].Services[svcId1], ShouldNotBeNil)
			So(state.Servers[hostname].Services[svcId2], ShouldNotBeNil)
			So(state.Servers[hostname].Services[svcId1].ID, ShouldEqual, svcId1)
			So(state.Servers[hostname].Services[svcId2].ID, ShouldEqual, svcId2)
		})

		Convey("All of the tombstones are serialized into the channel", func() {
			go func() { quit <- true }()
			junk := service.Service{ ID: "runs", Hostname: hostname, Updated: baseTime }
			state.AddServiceEntry(junk)
			state.AddServiceEntry(service1)
			state.AddServiceEntry(service2)
			go state.BroadcastTombstones(broadcasts, containerFn, quit)

			junk2 := service.Service{ ID: "runs", Hostname: hostname, Updated: baseTime }
			junk2.Status = service.TOMBSTONE
			//jsonStr, _ := json.Marshal(junk2)

			readBroadcasts := <-broadcasts
			So(len(readBroadcasts), ShouldEqual, 2) // 2 per service
			//SoSkip(string(readBroadcasts[0]), ShouldEqual, string(jsonStr))
			//SoSkip(string(readBroadcasts[1]), ShouldEqual, string(jsonStr))
		})

		Convey("Services that are still alive are not tombstoned", func() {
			go func() { quit <- true }()
			state.AddServiceEntry(service1)
			state.AddServiceEntry(service2)
			go state.BroadcastTombstones(broadcasts, containerFn, quit)

			readBroadcasts := <-broadcasts
			So(len(readBroadcasts), ShouldEqual, 0)
		})

		Reset(func() {
			broadcasts = make(chan [][]byte)
		})
	})
}

func ShouldBeTheSameTimeAs(actual interface{}, expected ...interface{}) string {
    wanted := expected[0].(time.Time)
    got    := actual.(time.Time)

    if !got.Equal(wanted) {
        return "expected:\n" + fmt.Sprintf("%#v", wanted) + "\n\ngot:\n" + fmt.Sprintf("%#v", got)
    }

    return ""
}
<commit_msg>Match the JSON with a regex since timestamps change.<commit_after>package services_state

import (
	"encoding/json"
	"fmt"
	"testing"
	"regexp"
	"time"

	. "github.com/smartystreets/goconvey/convey"
	"github.com/newrelic/bosun/service"
)

var hostname = "shakespeare"
var anotherHostname = "chaucer"

func Test_NewServer(t *testing.T) {

	Convey("Invoking NewServer()", t, func() {
		Convey("Returns a server with the correct name", func() {
			server := NewServer(hostname)
			So(server.Name, ShouldEqual, hostname)
		})

		Convey("Initializes the map", func() {
			server := NewServer(hostname)
			So(server.Services, ShouldNotBeNil)
		})

		Convey("Initializes the time", func() {
			server := NewServer(hostname)
			So(server.LastUpdated, ShouldBeTheSameTimeAs, time.Unix(0, 0))
		})
	})
}

func Test_NewServicesState(t *testing.T) {
	Convey("Invoking NewServicesState()", t, func() {

		Convey("Initializes the Servers map", func() {
			state := NewServicesState()
			So(state.Servers, ShouldNotBeNil)
		})

	})
}

func Test_ServicesStateWithData(t *testing.T) {

	Convey("When working with data", t, func() {
		state := NewServicesState()
		state.Servers[hostname] = NewServer(hostname)

		baseTime := time.Now().UTC()

		svc := service.Service{
			ID: "deadbeef123",
			Name: "radical_service",
			Image: "101deadbeef",
			Created: baseTime,
			Hostname: anotherHostname,
			Updated: baseTime,
			Status: service.ALIVE,
		}

		Convey("Encode() generates JSON that we can Decode()", func() {
			decoded, err := Decode(state.Encode())

			So(err, ShouldBeNil)
			So(decoded.Servers[hostname].Name, ShouldEqual, hostname)
			So(len(decoded.Servers), ShouldEqual, 1)
		})

		Convey("Decode() returns an error when handed junk", func() {
			result, err := Decode([]byte("asdf"))

			So(result.Servers, ShouldBeEmpty)
			So(err, ShouldNotBeNil)
		})

		Convey("HasServer() is true when a server exists", func() {
			So(state.HasServer(hostname), ShouldBeTrue)
		})

		Convey("HasServer() is false when a server is missing", func() {
			So(state.HasServer("junk"), ShouldBeFalse)
		})

		Convey("AddServiceEntry()", func() {
			Convey("Merges in a new service", func() {
				So(state.HasServer(anotherHostname), ShouldBeFalse)

				state.AddServiceEntry(svc)

				So(state.HasServer(anotherHostname), ShouldBeTrue)
				So(state.Servers[anotherHostname].Services[svc.ID], ShouldNotBeNil)
			})

			Convey("Doesn't merge a stale service", func() {
				state.AddServiceEntry(svc)

				staleService := service.Service{
					ID: "deadbeef123",
					Name: "stale_service",
					Image: "stale",
					Created: baseTime,
					Hostname: anotherHostname,
					Updated: baseTime.Add(0 - 1 * time.Minute),
					Status: service.ALIVE,
				}

				state.AddServiceEntry(staleService)

				So(state.HasServer(anotherHostname), ShouldBeTrue)
				So(state.Servers[anotherHostname].Services[svc.ID].Updated,
					ShouldBeTheSameTimeAs, baseTime)
				So(state.Servers[anotherHostname].Services[svc.ID].Image,
					ShouldEqual, "101deadbeef")
			})

			Convey("Updates the LastUpdated time for the server", func() {
				newDate := svc.Updated.AddDate(0, 0, 5)
				svc.Updated = newDate
				state.AddServiceEntry(svc)

				So(state.Servers[anotherHostname].LastUpdated, ShouldBeTheSameTimeAs, newDate)
			})
		})

		Convey("Format() pretty-prints the state even without a Memberlist", func() {
			formatted := state.Format(nil)

			So(formatted, ShouldNotBeNil)
		})

		Reset(func() {
			state = NewServicesState()
			state.Servers[hostname] = NewServer(hostname)
		})
	})
}

func Test_Broadcasts(t *testing.T) {

	Convey("When Broadcasting services", t, func() {
		state := NewServicesState()
		state.Servers[hostname] = NewServer(hostname)
		broadcasts := make(chan [][]byte)
		quit       := make(chan bool)
		svcId1     := "deadbeef123"
		svcId2     := "deadbeef101"
		baseTime   := time.Now().UTC().Round(time.Second)

		service1 := service.Service{ ID: svcId1, Hostname: hostname, Updated: baseTime }
		service2 := service.Service{ ID: svcId2, Hostname: hostname, Updated: baseTime }
		services := []service.Service{ service1, service2 }

		containerFn := func() []service.Service {
			return services
		}

		state.HostnameFn = func() (string, error) { return hostname, nil }

		Convey("New services are serialized into the channel", func() {
			go func() { quit <- true }()
			go state.BroadcastServices(broadcasts, containerFn, quit)

			json1, _ := json.Marshal(service1)
			json2, _ := json.Marshal(service2)

			readBroadcasts := <-broadcasts
			So(len(readBroadcasts), ShouldEqual, 2)
			So(string(readBroadcasts[0]), ShouldEqual, string(json1))
			So(string(readBroadcasts[1]), ShouldEqual, string(json2))
		})

		Convey("All of the services are added to state", func() {
			go func() { quit <- true }()
			go state.BroadcastServices(broadcasts, containerFn, quit)
			<-broadcasts // Block until we get a result

			So(state.Servers[hostname].Services[svcId1], ShouldNotBeNil)
			So(state.Servers[hostname].Services[svcId2], ShouldNotBeNil)
			So(state.Servers[hostname].Services[svcId1].ID, ShouldEqual, svcId1)
			So(state.Servers[hostname].Services[svcId2].ID, ShouldEqual, svcId2)
		})

		Convey("All of the tombstones are serialized into the channel", func() {
			go func() { quit <- true }()
			junk := service.Service{ ID: "runs", Hostname: hostname, Updated: baseTime }
			state.AddServiceEntry(junk)
			state.AddServiceEntry(service1)
			state.AddServiceEntry(service2)
			go state.BroadcastTombstones(broadcasts, containerFn, quit)

			readBroadcasts := <-broadcasts
			So(len(readBroadcasts), ShouldEqual, 2) // 2 per service
			// Match with regexes since the timestamp changes during tombstoning
			So(readBroadcasts[0], ShouldMatch, "^{\"ID\":\"runs\".*\"Status\":1}$")
			So(readBroadcasts[1], ShouldMatch, "^{\"ID\":\"runs\".*\"Status\":1}$")
		})

		Convey("Services that are still alive are not tombstoned", func() {
			go func() { quit <- true }()
			state.AddServiceEntry(service1)
			state.AddServiceEntry(service2)
			go state.BroadcastTombstones(broadcasts, containerFn, quit)

			readBroadcasts := <-broadcasts
			So(len(readBroadcasts), ShouldEqual, 0)
		})

		Reset(func() {
			broadcasts = make(chan [][]byte)
		})
	})
}

func ShouldBeTheSameTimeAs(actual interface{}, expected ...interface{}) string {
    wanted := expected[0].(time.Time)
    got    := actual.(time.Time)

    if !got.Equal(wanted) {
        return "expected:\n" + fmt.Sprintf("%#v", wanted) + "\n\ngot:\n" + fmt.Sprintf("%#v", got)
    }

    return ""
}

func ShouldMatch(actual interface{}, expected ...interface{}) string {
	wanted := expected[0].(string)
	got    := actual.([]byte)

	wantedRegexp := regexp.MustCompile(wanted)

	if !wantedRegexp.Match(got) {
		return "expected:\n" + fmt.Sprintf("%#v", wanted) + "\n\nto match:\n" + fmt.Sprintf("%v", string(got))
	}

	return ""
}
<|endoftext|>


<commit_before>package main

import (
	"bufio"
	"fmt"
	"os"
	"os/signal"
	"strings"

	"github.com/marcioAlmada/goremote/upnp"
	"github.com/tncardoso/gocurses"
)

type cursesApplication struct {
	screen *gocurses.Window
	application
}

func newCursesApplication() cursesApplication {
	return cursesApplication{}
}

func (cursesApplication) PromptPinCode() (str string) {
	reader := bufio.NewReader(os.Stdin)
	fmt.Print("Enter provided pin code: ")
	str, err := reader.ReadString('\n')
	nuke(err, "Could not read input.")
	str = strings.TrimRight(str, "\n\r")
	return
}

// Run executes the command line curses application
func (app cursesApplication) Run(client upnp.Client, keyMap, altKeyMap keyMap) (e error) {
	// handle process termination
	c := make(chan os.Signal, 1)
	signal.Notify(c, os.Interrupt)
	go func() {
		<-c
		gocurses.End()
		os.Exit(1)
	}()
	// add alternative key bidings
	keyMap.Merge(altKeyMap)
	// env config
	os.Setenv("ESCDELAY", "0")
	// curses config
	gocurses.CursSet(0)
	app.screen = gocurses.Initscr()
	defer gocurses.End()
	app.screen.Keypad(true)   // enable keypad support
	app.screen.Scrollok(true) // infinite screen
	gocurses.Noecho()         // avoid char leak of unmapped keys
	// run the REPL!
	app.screen.Addstr("> Ready to rumble!")
	for {
		ch := app.screen.Getch()
		if 4 == ch { // handles CTRL+D
			break
		}
		go func() {
			if key, ok := keyMap[ch]; ok {
				if _, _, e := client.SendCommand(key.Command); e == nil {
					app.screen.Addstr("\n", client.IP, "> ", key.Command)
				} else {
					app.screen.Addstr("\n", "ERROR: ", e)
				}
				app.screen.Refresh()
			}
		}()
	}
	return
}
<commit_msg>skips printing device IP if screen is not wide enough<commit_after>package main

import (
	"bufio"
	"fmt"
	"os"
	"os/signal"
	"strings"

	"github.com/marcioAlmada/goremote/upnp"
	"github.com/tncardoso/gocurses"
)

type cursesApplication struct {
	screen *gocurses.Window
	application
}

func newCursesApplication() cursesApplication {
	return cursesApplication{}
}

func (cursesApplication) PromptPinCode() (str string) {
	reader := bufio.NewReader(os.Stdin)
	fmt.Print("Enter provided pin code: ")
	str, err := reader.ReadString('\n')
	nuke(err, "Could not read input.")
	str = strings.TrimRight(str, "\n\r")
	return
}

// Run executes the command line curses application
func (app cursesApplication) Run(client upnp.Client, keyMap, altKeyMap keyMap) (e error) {
	c := make(chan os.Signal, 1) // handle process termination
	signal.Notify(c, os.Interrupt)
	go func() {
		<-c
		gocurses.End()
		os.Exit(1)
	}()
	// add alternative key bidings
	keyMap.Merge(altKeyMap)
	// env config
	os.Setenv("ESCDELAY", "0")
	// curses config
	gocurses.CursSet(0)
	app.screen = gocurses.Initscr()
	defer gocurses.End()
	app.screen.Keypad(true)   // enable keypad support
	app.screen.Scrollok(true) // infinite screen
	gocurses.Noecho()         // avoid char leak of unmapped keys
	// run the REPL!
	app.screen.Addstr("> Ready!")
	for {
		ch := app.screen.Getch()
		if 4 == ch { // handles CTRL+D
			break
		}
		go func() {
			if key, ok := keyMap[ch]; ok {
				if _, _, e := client.SendCommand(key.Command); e == nil {
					if _, windowx := app.screen.Getmaxyx(); windowx > 30 {
						app.screen.Addstr("\n", client.IP, "> ", key.Command)
					} else {
						app.screen.Addstr("\n> ", key.Command)
					}
				} else {
					app.screen.Addstr("\n", "ERROR: ", e)
				}
				app.screen.Refresh()
			}
		}()
	}
	return
}
<|endoftext|>


<commit_before>package integrationtests

import (
	"crypto/md5"
	"crypto/rand"
)

type dataManager struct {
	data []byte
	md5  []byte
}

func (m *dataManager) GenerateData(len int) error {
	m.data = make([]byte, len)
	_, err := rand.Read(m.data)
	if err != nil {
		return err
	}
	sum := md5.Sum(m.data)
	m.md5 = sum[:]
	return nil
}

func (m *dataManager) GetData() []byte {
	return m.data
}

func (m *dataManager) GetMD5() []byte {
	return m.md5
}
<commit_msg>use mathematical randomness in integration tests<commit_after>package integrationtests

import (
	"crypto/md5"
	"math/rand"
	"time"
)

type dataManager struct {
	data []byte
	md5  []byte
}

func (m *dataManager) GenerateData(len int) error {
	m.data = make([]byte, len)
	r := rand.New(rand.NewSource(int64(time.Now().Nanosecond())))
	_, err := r.Read(m.data)
	if err != nil {
		return err
	}
	sum := md5.Sum(m.data)
	m.md5 = sum[:]
	return nil
}

func (m *dataManager) GetData() []byte {
	return m.data
}

func (m *dataManager) GetMD5() []byte {
	return m.md5
}
<|endoftext|>