<commit_before>// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

/*
Package packages loads Go packages for inspection and analysis.

NOTE: THIS PACKAGE IS NOT YET READY FOR WIDESPREAD USE:
 - The interface is still being revised and minor changes are likely.
 - The implementation depends on the Go 1.11 go command;
   support for earlier versions will be added soon.
 - We intend to finalize the API before Go 1.11 is released.

The Load function takes as input a list of patterns and return a list of Package
structs describing individual packages matched by those patterns.
The LoadMode controls the amount of detail in the loaded packages.

Load passes most patterns directly to the underlying build tool,
but all patterns with the prefix "query=", where query is a
non-empty string of letters from [a-z], are reserved and may be
interpreted as query operators.

Only two query operators are currently supported, "file" and "pattern".

The query "file=path/to/file.go" matches the package or packages enclosing
the Go source file path/to/file.go.  For example "file=~/go/src/fmt/print.go"
might returns the packages "fmt" and "fmt [fmt.test]".

The query "pattern=string" causes "string" to be passed directly to
the underlying build tool. In most cases this is unnecessary,
but an application can use Load("pattern=" + x) as an escaping mechanism
to ensure that x is not interpreted as a query operator if it contains '='.

A third query "name=identifier" will be added soon.
It will match packages whose package declaration contains the specified identifier.
For example, "name=rand" would match the packages "math/rand" and "crypto/rand",
and "name=main" would match all executables.

All other query operators are reserved for future use and currently
cause Load to report an error.

The Package struct provides basic information about the package, including

  - ID, a unique identifier for the package in the returned set;
  - GoFiles, the names of the package's Go source files;
  - Imports, a map from source import strings to the Packages they name;
  - Types, the type information for the package's exported symbols;
  - Syntax, the parsed syntax trees for the package's source code; and
  - TypeInfo, the result of a complete type-check of the package syntax trees.

(See the documentation for type Package for the complete list of fields
and more detailed descriptions.)

For example,

	Load(nil, "bytes", "unicode...")

returns four Package structs describing the standard library packages
bytes, unicode, unicode/utf16, and unicode/utf8. Note that one pattern
can match multiple packages and that a package might be matched by
multiple patterns: in general it is not possible to determine which
packages correspond to which patterns.

Note that the list returned by Load contains only the packages matched
by the patterns. Their dependencies can be found by walking the import
graph using the Imports fields.

The Load function can be configured by passing a pointer to a Config as
the first argument. A nil Config is equivalent to the zero Config, which
causes Load to run in LoadFiles mode, collecting minimal information.
See the documentation for type Config for details.

As noted earlier, the Config.Mode controls the amount of detail
reported about the loaded packages, with each mode returning all the data of the
previous mode with some extra added. See the documentation for type LoadMode
for details.

Most tools should pass their command-line arguments (after any flags)
uninterpreted to the loader, so that the loader can interpret them
according to the conventions of the underlying build system.
See the Example function for typical usage.

*/
package packages // import "golang.org/x/tools/go/packages"

/*

Motivation and design considerations

The new package's design solves problems addressed by two existing
packages: go/build, which locates and describes packages, and
golang.org/x/tools/go/loader, which loads, parses and type-checks them.
The go/build.Package structure encodes too much of the 'go build' way
of organizing projects, leaving us in need of a data type that describes a
package of Go source code independent of the underlying build system.
We wanted something that works equally well with go build and vgo, and
also other build systems such as Bazel and Blaze, making it possible to
construct analysis tools that work in all these environments.
Tools such as errcheck and staticcheck were essentially unavailable to
the Go community at Google, and some of Google's internal tools for Go
are unavailable externally.
This new package provides a uniform way to obtain package metadata by
querying each of these build systems, optionally supporting their
preferred command-line notations for packages, so that tools integrate
neatly with users' build environments. The Metadata query function
executes an external query tool appropriate to the current workspace.

Loading packages always returns the complete import graph "all the way down",
even if all you want is information about a single package, because the query
mechanisms of all the build systems we currently support ({go,vgo} list, and
blaze/bazel aspect-based query) cannot provide detailed information
about one package without visiting all its dependencies too, so there is
no additional asymptotic cost to providing transitive information.
(This property might not be true of a hypothetical 5th build system.)

In calls to TypeCheck, all initial packages, and any package that
transitively depends on one of them, must be loaded from source.
Consider A->B->C->D->E: if A,C are initial, A,B,C must be loaded from
source; D may be loaded from export data, and E may not be loaded at all
(though it's possible that D's export data mentions it, so a
types.Package may be created for it and exposed.)

The old loader had a feature to suppress type-checking of function
bodies on a per-package basis, primarily intended to reduce the work of
obtaining type information for imported packages. Now that imports are
satisfied by export data, the optimization no longer seems necessary.

Despite some early attempts, the old loader did not exploit export data,
instead always using the equivalent of WholeProgram mode. This was due
to the complexity of mixing source and export data packages (now
resolved by the upward traversal mentioned above), and because export data
files were nearly always missing or stale. Now that 'go build' supports
caching, all the underlying build systems can guarantee to produce
export data in a reasonable (amortized) time.

Test "main" packages synthesized by the build system are now reported as
first-class packages, avoiding the need for clients (such as go/ssa) to
reinvent this generation logic.

One way in which go/packages is simpler than the old loader is in its
treatment of in-package tests. In-package tests are packages that
consist of all the files of the library under test, plus the test files.
The old loader constructed in-package tests by a two-phase process of
mutation called "augmentation": first it would construct and type check
all the ordinary library packages and type-check the packages that
depend on them; then it would add more (test) files to the package and
type-check again. This two-phase approach had four major problems:
1) in processing the tests, the loader modified the library package,
   leaving no way for a client application to see both the test
   package and the library package; one would mutate into the other.
2) because test files can declare additional methods on types defined in
   the library portion of the package, the dispatch of method calls in
   the library portion was affected by the presence of the test files.
   This should have been a clue that the packages were logically
   different.
3) this model of "augmentation" assumed at most one in-package test
   per library package, which is true of projects using 'go build',
   but not other build systems.
4) because of the two-phase nature of test processing, all packages that
   import the library package had to be processed before augmentation,
   forcing a "one-shot" API and preventing the client from calling Load
   in several times in sequence as is now possible in WholeProgram mode.
   (TypeCheck mode has a similar one-shot restriction for a different reason.)

Early drafts of this package supported "multi-shot" operation.
Although it allowed clients to make a sequence of calls (or concurrent
calls) to Load, building up the graph of Packages incrementally,
it was of marginal value: it complicated the API
(since it allowed some options to vary across calls but not others),
it complicated the implementation,
it cannot be made to work in Types mode, as explained above,
and it was less efficient than making one combined call (when this is possible).
Among the clients we have inspected, none made multiple calls to load
but could not be easily and satisfactorily modified to make only a single call.
However, applications changes may be required.
For example, the ssadump command loads the user-specified packages
and in addition the runtime package.  It is tempting to simply append
"runtime" to the user-provided list, but that does not work if the user
specified an ad-hoc package such as [a.go b.go].
Instead, ssadump no longer requests the runtime package,
but seeks it among the dependencies of the user-specified packages,
and emits an error if it is not found.

Overlays: the ParseFile hook in the API permits clients to vary the way
in which ASTs are obtained from filenames; the default implementation is
based on parser.ParseFile. This features enables editor-integrated tools
that analyze the contents of modified but unsaved buffers: rather than
read from the file system, a tool can read from an archive of modified
buffers provided by the editor.
This approach has its limits. Because package metadata is obtained by
fork/execing an external query command for each build system, we can
fake only the file contents seen by the parser, type-checker, and
application, but not by the metadata query, so, for example:
- additional imports in the fake file will not be described by the
  metadata, so the type checker will fail to load imports that create
  new dependencies.
- in TypeCheck mode, because export data is produced by the query
  command, it will not reflect the fake file contents.
- this mechanism cannot add files to a package without first saving them.

Questions & Tasks

- Add GOARCH/GOOS?
  They are not portable concepts, but could be made portable.
  Our goal has been to allow users to express themselves using the conventions
  of the underlying build system: if the build system honors GOARCH
  during a build and during a metadata query, then so should
  applications built atop that query mechanism.
  Conversely, if the target architecture of the build is determined by
  command-line flags, the application can pass the relevant
  flags through to the build system using a command such as:
    myapp -query_flag="--cpu=amd64" -query_flag="--os=darwin"
  However, this approach is low-level, unwieldy, and non-portable.
  GOOS and GOARCH seem important enough to warrant a dedicated option.

- How should we handle partial failures such as a mixture of good and
  malformed patterns, existing and non-existent packages, successful and
  failed builds, import failures, import cycles, and so on, in a call to
  Load?

- Support bazel, blaze, and go1.10 list, not just go1.11 list.

- Handle (and test) various partial success cases, e.g.
  a mixture of good packages and:
  invalid patterns
  nonexistent packages
  empty packages
  packages with malformed package or import declarations
  unreadable files
  import cycles
  other parse errors
  type errors
  Make sure we record errors at the correct place in the graph.

- Missing packages among initial arguments are not reported.
  Return bogus packages for them, like golist does.

- "undeclared name" errors (for example) are reported out of source file
  order. I suspect this is due to the breadth-first resolution now used
  by go/types. Is that a bug? Discuss with gri.

*/
<commit_msg>go/packages: remove scary warning on documentation<commit_after>// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

/*
Package packages loads Go packages for inspection and analysis.

Note: Though this package is ready for widespread use, we may make minor
breaking changes if absolutely necessary. Any such change will be
announced on golang-tools@ at least one week before it is committed. No
more breaking changes will be made after December 1, 2018.

The Load function takes as input a list of patterns and return a list of Package
structs describing individual packages matched by those patterns.
The LoadMode controls the amount of detail in the loaded packages.

Load passes most patterns directly to the underlying build tool,
but all patterns with the prefix "query=", where query is a
non-empty string of letters from [a-z], are reserved and may be
interpreted as query operators.

Only two query operators are currently supported, "file" and "pattern".

The query "file=path/to/file.go" matches the package or packages enclosing
the Go source file path/to/file.go.  For example "file=~/go/src/fmt/print.go"
might returns the packages "fmt" and "fmt [fmt.test]".

The query "pattern=string" causes "string" to be passed directly to
the underlying build tool. In most cases this is unnecessary,
but an application can use Load("pattern=" + x) as an escaping mechanism
to ensure that x is not interpreted as a query operator if it contains '='.

A third query "name=identifier" will be added soon.
It will match packages whose package declaration contains the specified identifier.
For example, "name=rand" would match the packages "math/rand" and "crypto/rand",
and "name=main" would match all executables.

All other query operators are reserved for future use and currently
cause Load to report an error.

The Package struct provides basic information about the package, including

  - ID, a unique identifier for the package in the returned set;
  - GoFiles, the names of the package's Go source files;
  - Imports, a map from source import strings to the Packages they name;
  - Types, the type information for the package's exported symbols;
  - Syntax, the parsed syntax trees for the package's source code; and
  - TypeInfo, the result of a complete type-check of the package syntax trees.

(See the documentation for type Package for the complete list of fields
and more detailed descriptions.)

For example,

	Load(nil, "bytes", "unicode...")

returns four Package structs describing the standard library packages
bytes, unicode, unicode/utf16, and unicode/utf8. Note that one pattern
can match multiple packages and that a package might be matched by
multiple patterns: in general it is not possible to determine which
packages correspond to which patterns.

Note that the list returned by Load contains only the packages matched
by the patterns. Their dependencies can be found by walking the import
graph using the Imports fields.

The Load function can be configured by passing a pointer to a Config as
the first argument. A nil Config is equivalent to the zero Config, which
causes Load to run in LoadFiles mode, collecting minimal information.
See the documentation for type Config for details.

As noted earlier, the Config.Mode controls the amount of detail
reported about the loaded packages, with each mode returning all the data of the
previous mode with some extra added. See the documentation for type LoadMode
for details.

Most tools should pass their command-line arguments (after any flags)
uninterpreted to the loader, so that the loader can interpret them
according to the conventions of the underlying build system.
See the Example function for typical usage.

*/
package packages // import "golang.org/x/tools/go/packages"

/*

Motivation and design considerations

The new package's design solves problems addressed by two existing
packages: go/build, which locates and describes packages, and
golang.org/x/tools/go/loader, which loads, parses and type-checks them.
The go/build.Package structure encodes too much of the 'go build' way
of organizing projects, leaving us in need of a data type that describes a
package of Go source code independent of the underlying build system.
We wanted something that works equally well with go build and vgo, and
also other build systems such as Bazel and Blaze, making it possible to
construct analysis tools that work in all these environments.
Tools such as errcheck and staticcheck were essentially unavailable to
the Go community at Google, and some of Google's internal tools for Go
are unavailable externally.
This new package provides a uniform way to obtain package metadata by
querying each of these build systems, optionally supporting their
preferred command-line notations for packages, so that tools integrate
neatly with users' build environments. The Metadata query function
executes an external query tool appropriate to the current workspace.

Loading packages always returns the complete import graph "all the way down",
even if all you want is information about a single package, because the query
mechanisms of all the build systems we currently support ({go,vgo} list, and
blaze/bazel aspect-based query) cannot provide detailed information
about one package without visiting all its dependencies too, so there is
no additional asymptotic cost to providing transitive information.
(This property might not be true of a hypothetical 5th build system.)

In calls to TypeCheck, all initial packages, and any package that
transitively depends on one of them, must be loaded from source.
Consider A->B->C->D->E: if A,C are initial, A,B,C must be loaded from
source; D may be loaded from export data, and E may not be loaded at all
(though it's possible that D's export data mentions it, so a
types.Package may be created for it and exposed.)

The old loader had a feature to suppress type-checking of function
bodies on a per-package basis, primarily intended to reduce the work of
obtaining type information for imported packages. Now that imports are
satisfied by export data, the optimization no longer seems necessary.

Despite some early attempts, the old loader did not exploit export data,
instead always using the equivalent of WholeProgram mode. This was due
to the complexity of mixing source and export data packages (now
resolved by the upward traversal mentioned above), and because export data
files were nearly always missing or stale. Now that 'go build' supports
caching, all the underlying build systems can guarantee to produce
export data in a reasonable (amortized) time.

Test "main" packages synthesized by the build system are now reported as
first-class packages, avoiding the need for clients (such as go/ssa) to
reinvent this generation logic.

One way in which go/packages is simpler than the old loader is in its
treatment of in-package tests. In-package tests are packages that
consist of all the files of the library under test, plus the test files.
The old loader constructed in-package tests by a two-phase process of
mutation called "augmentation": first it would construct and type check
all the ordinary library packages and type-check the packages that
depend on them; then it would add more (test) files to the package and
type-check again. This two-phase approach had four major problems:
1) in processing the tests, the loader modified the library package,
   leaving no way for a client application to see both the test
   package and the library package; one would mutate into the other.
2) because test files can declare additional methods on types defined in
   the library portion of the package, the dispatch of method calls in
   the library portion was affected by the presence of the test files.
   This should have been a clue that the packages were logically
   different.
3) this model of "augmentation" assumed at most one in-package test
   per library package, which is true of projects using 'go build',
   but not other build systems.
4) because of the two-phase nature of test processing, all packages that
   import the library package had to be processed before augmentation,
   forcing a "one-shot" API and preventing the client from calling Load
   in several times in sequence as is now possible in WholeProgram mode.
   (TypeCheck mode has a similar one-shot restriction for a different reason.)

Early drafts of this package supported "multi-shot" operation.
Although it allowed clients to make a sequence of calls (or concurrent
calls) to Load, building up the graph of Packages incrementally,
it was of marginal value: it complicated the API
(since it allowed some options to vary across calls but not others),
it complicated the implementation,
it cannot be made to work in Types mode, as explained above,
and it was less efficient than making one combined call (when this is possible).
Among the clients we have inspected, none made multiple calls to load
but could not be easily and satisfactorily modified to make only a single call.
However, applications changes may be required.
For example, the ssadump command loads the user-specified packages
and in addition the runtime package.  It is tempting to simply append
"runtime" to the user-provided list, but that does not work if the user
specified an ad-hoc package such as [a.go b.go].
Instead, ssadump no longer requests the runtime package,
but seeks it among the dependencies of the user-specified packages,
and emits an error if it is not found.

Overlays: the ParseFile hook in the API permits clients to vary the way
in which ASTs are obtained from filenames; the default implementation is
based on parser.ParseFile. This features enables editor-integrated tools
that analyze the contents of modified but unsaved buffers: rather than
read from the file system, a tool can read from an archive of modified
buffers provided by the editor.
This approach has its limits. Because package metadata is obtained by
fork/execing an external query command for each build system, we can
fake only the file contents seen by the parser, type-checker, and
application, but not by the metadata query, so, for example:
- additional imports in the fake file will not be described by the
  metadata, so the type checker will fail to load imports that create
  new dependencies.
- in TypeCheck mode, because export data is produced by the query
  command, it will not reflect the fake file contents.
- this mechanism cannot add files to a package without first saving them.

Questions & Tasks

- Add GOARCH/GOOS?
  They are not portable concepts, but could be made portable.
  Our goal has been to allow users to express themselves using the conventions
  of the underlying build system: if the build system honors GOARCH
  during a build and during a metadata query, then so should
  applications built atop that query mechanism.
  Conversely, if the target architecture of the build is determined by
  command-line flags, the application can pass the relevant
  flags through to the build system using a command such as:
    myapp -query_flag="--cpu=amd64" -query_flag="--os=darwin"
  However, this approach is low-level, unwieldy, and non-portable.
  GOOS and GOARCH seem important enough to warrant a dedicated option.

- How should we handle partial failures such as a mixture of good and
  malformed patterns, existing and non-existent packages, successful and
  failed builds, import failures, import cycles, and so on, in a call to
  Load?

- Support bazel, blaze, and go1.10 list, not just go1.11 list.

- Handle (and test) various partial success cases, e.g.
  a mixture of good packages and:
  invalid patterns
  nonexistent packages
  empty packages
  packages with malformed package or import declarations
  unreadable files
  import cycles
  other parse errors
  type errors
  Make sure we record errors at the correct place in the graph.

- Missing packages among initial arguments are not reported.
  Return bogus packages for them, like golist does.

- "undeclared name" errors (for example) are reported out of source file
  order. I suspect this is due to the breadth-first resolution now used
  by go/types. Is that a bug? Discuss with gri.

*/
<|endoftext|>


<commit_before>/*
Package saltpack is an implementation of the Saltpack message format. Saltpack
is a very light wrapper around Dan Berstein's famous NaCl library. It adds
support for longer messages, streaming input and output of data, multiple
recipients for encrypted messages, and a reasonable armoring format. We intend
Saltpack as a replacement for the PGP messaging format, as it can be used in
many of the same circumstances.  However, it is designed to be: (1) simpler;
(2) easier to implement; (3) judicious (perhaps judgmental) in its crypto
usage; (4) fully modern (no CBC option here); (5) high performance; (6) less
bug-prone; (7) generally unwilling to output unauthenticated data; and (8)
easier to compose with other software in any manner of languages or platforms.

Saltpack makes no attempt to manage keys. We assume the wrapping application
has a story for key management.

Saltpack supports three modes of operation: encrypted messages, attached
signatures, and detached signatures. An attached signature contains a message
and a signature that authenticates it. A detached signature contains just the
signature, and assumes an independent delievery mechanism for the file
(this might come up when distributing an ISO an separate signature of it).

Saltpack has two encoding modes: binary and armored. In armored mode, saltpack
outputs in Base62-encoding, suitable for publication into any manner of Web
settings without fear of markup-caused mangling.

This saltpack library implementation supports two API patterns: streaming and
all-at-once. The former is useful for large files that can't fit into memory;
the latter is more convenient. Both produce the same output.
*/
package saltpack
<commit_msg>doc encoding for saltpack<commit_after>/*
Package saltpack is an implementation of the Saltpack message format. Saltpack
is a very light wrapper around Dan Berstein's famous NaCl library. It adds
support for longer messages, streaming input and output of data, multiple
recipients for encrypted messages, and a reasonable armoring format. We intend
Saltpack as a replacement for the PGP messaging format, as it can be used in
many of the same circumstances.  However, it is designed to be: (1) simpler;
(2) easier to implement; (3) judicious (perhaps judgmental) in its crypto
usage; (4) fully modern (no CBC option here); (5) high performance; (6) less
bug-prone; (7) generally unwilling to output unauthenticated data; and (8)
easier to compose with other software in any manner of languages or platforms.

Key Management

Saltpack makes no attempt to manage keys. We assume the wrapping application
has a story for key management.

Modes of Operation

Saltpack supports three modes of operation: encrypted messages, attached
signatures, and detached signatures. An attached signature contains a message
and a signature that authenticates it. A detached signature contains just the
signature, and assumes an independent delievery mechanism for the file
(this might come up when distributing an ISO an separate signature of it).

Encoding

Saltpack has two encoding modes: binary and armored. In armored mode, saltpack
outputs in Base62-encoding, suitable for publication into any manner of Web
settings without fear of markup-caused mangling.

API

This saltpack library implementation supports two API patterns: streaming and
all-at-once. The former is useful for large files that can't fit into memory;
the latter is more convenient. Both produce the same output.

More Info

See https://saltpack.org

*/
package saltpack
<|endoftext|>


<commit_before>package main

import (
	"go/build"
	"io/ioutil"
	"os"
	"path/filepath"

	"github.com/posener/complete"
)

func predictPackages(a complete.Args) (prediction []string) {
	for {
		prediction = complete.PredictFilesSet(listPackages(a)).Predict(a)

		// if the number of prediction is not 1, we either have many results or
		// have no results, so we return it.
		if len(prediction) != 1 {
			return
		}

		// if the result is only one item, we might want to recursively check
		// for more accurate results.
		if prediction[0] == a.Last {
			return
		}

		// only try deeper, if the one item is a directory
		if stat, err := os.Stat(prediction[0]); err != nil || !stat.IsDir() {
			return
		}

		a.Last = prediction[0]
	}
}

func listPackages(a complete.Args) (dirctories []string) {
	dir := a.Directory()
	complete.Log("listing packages in %s", dir)
	// import current directory
	pkg, err := build.ImportDir(dir, 0)
	if err != nil {
		complete.Log("failed importing directory %s: %s", dir, err)
		return
	}
	dirctories = append(dirctories, pkg.Dir)

	// import subdirectories
	files, err := ioutil.ReadDir(dir)
	if err != nil {
		complete.Log("failed reading directory %s: %s", dir, err)
		return
	}
	for _, f := range files {
		if !f.IsDir() {
			continue
		}
		pkg, err := build.ImportDir(filepath.Join(dir, f.Name()), 0)
		if err != nil {
			complete.Log("failed importing subdirectory %s: %s", filepath.Join(dir, f.Name()), err)
			continue
		}
		dirctories = append(dirctories, pkg.Dir)
	}
	return
}
<commit_msg>gocomplete: improve package completion<commit_after>package main

import (
	"go/build"
	"io/ioutil"
	"path/filepath"

	"github.com/posener/complete"
)

// predictPackages completes packages in the directory pointed by a.Last
// and packages that are one level below that package.
func predictPackages(a complete.Args) (prediction []string) {
	prediction = complete.PredictFilesSet(listPackages(a.Directory())).Predict(a)
	if len(prediction) != 1 {
		return
	}
	return complete.PredictFilesSet(listPackages(prediction[0])).Predict(a)
}

// listPackages looks in current pointed dir and in all it's direct sub-packages
// and return a list of paths to go packages.
func listPackages(dir string) (directories []string) {
	// add subdirectories
	files, err := ioutil.ReadDir(dir)
	if err != nil {
		complete.Log("failed reading directory %s: %s", dir, err)
		return
	}

	// build paths array
	paths := make([]string, 0, len(files)+1)
	for _, f := range files {
		if f.IsDir() {
			paths = append(paths, filepath.Join(dir, f.Name()))
		}
	}
	paths = append(paths, dir)

	// import packages according to given paths
	for _, p := range paths {
		pkg, err := build.ImportDir(p, 0)
		if err != nil {
			complete.Log("failed importing directory %s: %s", p, err)
			continue
		}
		directories = append(directories, pkg.Dir)
	}
	return
}
<|endoftext|>


<commit_before>package gocroaring

import (
	"fmt"
	"testing"
)

func TestSimpleCard(t *testing.T) {
	bitmap := New()
	for i := 100; i < 1000; i++ {
		bitmap.Add(uint32(i))
	}
	c := bitmap.Cardinality()
	fmt.Println("cardinality: ", c)
	if c != 900 {
		t.Error("Expected ", 900, ", got ", c)
	}
	bitmap.RunOptimize()
	if c != 900 {
		t.Error("Expected ", 900, ", got ", c)
	}
}

func TestNewWithVals(t *testing.T) {
	vals := []uint32{1, 2, 3, 6, 7, 8, 20, 44444}
	rb := New(vals...)
	for _, v := range vals {
		if !rb.Contains(v) {
			t.Errorf("expected %d from initialized values\n", v)
		}
	}
}

func TestAddMany(t *testing.T) {
	rb1 := New()
	sl := []uint32{1, 2, 3, 6, 7, 8, 20, 44444}
	rb1.Add(sl...)

	if int(rb1.Cardinality()) != len(sl) {
		t.Errorf("cardinality: expected %d, got %d", rb1.Cardinality(), len(sl))
	}
	if rb1.Contains(5) {
		t.Error("didn't expect to contain 5")
	}
	for _, v := range sl {
		if !rb1.Contains(v) {
			t.Errorf("expected to contain %d", v)
		}
	}
}

func TestFancier(t *testing.T) {
	rb1 := New()
	rb1.Add(1)
	rb1.Add(2)
	rb1.Add(3)
	rb1.Add(4)
	rb1.Add(5)
	rb1.Add(100)
	rb1.Add(1000)
	rb1.RunOptimize()
	rb2 := New()
	rb2.Add(3)
	rb2.Add(4)
	rb2.Add(1000)
	rb2.RunOptimize()
	rb3 := New()
	fmt.Println("Cardinality: ", rb1.Cardinality())
	if rb1.Cardinality() != 7 {
		t.Error("Bad card")
	}
	if !rb1.Contains(3) {
		t.Error("should contain it")
	}
	rb1.And(rb2)
	fmt.Println(rb1)
	rb3.Add(5)
	rb3.Or(rb1)
	fmt.Println(rb3.ToArray())
	fmt.Println(rb3)
	rb4 := FastOr(rb1, rb2, rb3)
	fmt.Println(rb4)
	// next we include an example of serialization
	buf := make([]byte, rb1.SerializedSizeInBytes())
	rb1.Write(buf) // we omit error handling
	newrb, _ := Read(buf)
	if rb1.Equals(newrb) {
		fmt.Println("I wrote the content to a byte stream and read it back.")
	} else {
		t.Error("Bad read")
	}
}

func TestStats(t *testing.T) {

	rb := New()
	rb.Add(1, 2, 3, 4, 6, 7)
	rb.Add(999991, 999992, 999993, 999994, 999996, 999997)

	stats := rb.Stats()
	if stats["cardinality"] != rb.Cardinality() {
		t.Errorf("cardinality: expected %d got %d\n", rb.Cardinality(), stats["cardinality"])
	}

	if stats["n_containers"] != 2 {
		t.Errorf("n_containers: expected %d got %d\n", 2, stats["n_containers"], stats)
	}
	if stats["n_array_containers"] != 2 {
		t.Errorf("n_array_containers: expected %d got %d\n", 2, stats["n_array_containers"], stats)
	}
	for _, c := range []string{"n_run_containers", "n_bitmap_containers"} {
		if stats[c] != 0 {
			t.Errorf("%s: expected 0 got %d\n", 2, c, stats[c])
		}
	}
}
<commit_msg>Added memory stress test<commit_after>package gocroaring

import (
	"fmt"
	"testing"
)

// go test -run MemoryUsage
func TestMemoryUsage(t *testing.T) {
  bitmap := New()
  for i:= 0; i < 1000000; i++ {
    bitmap.Add(uint32(i)*10)
  }
  sb := bitmap.SerializedSizeInBytes()
  memory_alloc := 8*1024*1024*1024
  howmany := (memory_alloc + sb - 1)/ sb
  fmt.Println("size in MB of one bitmap ", sb/(1024*1024), "; number of copies = ", howmany, "; total alloc: ", howmany * sb / (1024*1024*1024), "GB")
  for i:= 0; i < howmany; i++ {
    y := bitmap.Clone()
    _ = y
  }
}

func TestSimpleCard(t *testing.T) {
	bitmap := New()
	for i := 100; i < 1000; i++ {
		bitmap.Add(uint32(i))
	}
	c := bitmap.Cardinality()
	fmt.Println("cardinality: ", c)
	if c != 900 {
		t.Error("Expected ", 900, ", got ", c)
	}
	bitmap.RunOptimize()
	if c != 900 {
		t.Error("Expected ", 900, ", got ", c)
	}
}

func TestNewWithVals(t *testing.T) {
	vals := []uint32{1, 2, 3, 6, 7, 8, 20, 44444}
	rb := New(vals...)
	for _, v := range vals {
		if !rb.Contains(v) {
			t.Errorf("expected %d from initialized values\n", v)
		}
	}
}

func TestAddMany(t *testing.T) {
	rb1 := New()
	sl := []uint32{1, 2, 3, 6, 7, 8, 20, 44444}
	rb1.Add(sl...)

	if int(rb1.Cardinality()) != len(sl) {
		t.Errorf("cardinality: expected %d, got %d", rb1.Cardinality(), len(sl))
	}
	if rb1.Contains(5) {
		t.Error("didn't expect to contain 5")
	}
	for _, v := range sl {
		if !rb1.Contains(v) {
			t.Errorf("expected to contain %d", v)
		}
	}
}

func TestFancier(t *testing.T) {
	rb1 := New()
	rb1.Add(1)
	rb1.Add(2)
	rb1.Add(3)
	rb1.Add(4)
	rb1.Add(5)
	rb1.Add(100)
	rb1.Add(1000)
	rb1.RunOptimize()
	rb2 := New()
	rb2.Add(3)
	rb2.Add(4)
	rb2.Add(1000)
	rb2.RunOptimize()
	rb3 := New()
	fmt.Println("Cardinality: ", rb1.Cardinality())
	if rb1.Cardinality() != 7 {
		t.Error("Bad card")
	}
	if !rb1.Contains(3) {
		t.Error("should contain it")
	}
	rb1.And(rb2)
	fmt.Println(rb1)
	rb3.Add(5)
	rb3.Or(rb1)
	fmt.Println(rb3.ToArray())
	fmt.Println(rb3)
	rb4 := FastOr(rb1, rb2, rb3)
	fmt.Println(rb4)
	// next we include an example of serialization
	buf := make([]byte, rb1.SerializedSizeInBytes())
	rb1.Write(buf) // we omit error handling
	newrb, _ := Read(buf)
	if rb1.Equals(newrb) {
		fmt.Println("I wrote the content to a byte stream and read it back.")
	} else {
		t.Error("Bad read")
	}
}

func TestStats(t *testing.T) {

	rb := New()
	rb.Add(1, 2, 3, 4, 6, 7)
	rb.Add(999991, 999992, 999993, 999994, 999996, 999997)

	stats := rb.Stats()
	if stats["cardinality"] != rb.Cardinality() {
		t.Errorf("cardinality: expected %d got %d\n", rb.Cardinality(), stats["cardinality"])
	}

	if stats["n_containers"] != 2 {
		t.Errorf("n_containers: expected %d got %d\n", 2, stats["n_containers"], stats)
	}
	if stats["n_array_containers"] != 2 {
		t.Errorf("n_array_containers: expected %d got %d\n", 2, stats["n_array_containers"], stats)
	}
	for _, c := range []string{"n_run_containers", "n_bitmap_containers"} {
		if stats[c] != 0 {
			t.Errorf("%s: expected 0 got %d\n", 2, c, stats[c])
		}
	}
}
<|endoftext|>


<commit_before>package golist

import (
	"bufio"
	"encoding/json"
	"fmt"
	"io"
	//	"fmt"
	"log"
	"os"
	"os/exec"
	"sort"
	"strings"
	"text/template"
)

// Context is similar to
// // https://github.com/golang/go/blob/master/src/cmd/go/context.go
type Context struct {
	GOARCH        string   // target architecture
	GOOS          string   // target operating system
	GOROOT        string   // Go root
	GOPATH        string   // Go path
	CgoEnabled    bool     // whether cgo can be used
	UseAllFiles   bool     // use files regardless of +build lines, file names
	Compiler      string   // compiler to assume when computing target paths
	BuildTags     []string // build constraints to match in +build lines
	ReleaseTags   []string // releases the current release is compatible with
	InstallSuffix string   // suffix to use in the name of the install dir
}

// A PackageError describes an error loading information about a package.
type PackageError struct {
	ImportStack []string // shortest path from package named on command line to this one
	Pos         string   // position of error
	Err         string   // the error itself
}

// Package is copy of the Package struct as listed in https://golang.org/src/cmd/go/list.go
//  oddly not exported in golang
type Package struct {
	Dir           string // directory containing package sources
	ImportPath    string // import path of package in dir
	ImportComment string // path in import comment on package statement
	Name          string // package name
	Doc           string // package documentation string
	Target        string // install path
	Shlib         string // the shared library that contains this package (only set when -linkshared)
	Goroot        bool   // is this package in the Go root?
	Standard      bool   // is this package part of the standard Go library?
	Stale         bool   // would 'go install' do anything for this package?
	Root          string // Go root or Go path dir containing this package

	// Source files
	GoFiles        []string // .go source files (excluding CgoFiles, TestGoFiles, XTestGoFiles)
	CgoFiles       []string // .go sources files that import "C"
	IgnoredGoFiles []string // .go sources ignored due to build constraints
	CFiles         []string // .c source files
	CXXFiles       []string // .cc, .cxx and .cpp source files
	MFiles         []string // .m source files
	HFiles         []string // .h, .hh, .hpp and .hxx source files
	SFiles         []string // .s source files
	SwigFiles      []string // .swig files
	SwigCXXFiles   []string // .swigcxx files
	SysoFiles      []string // .syso object files to add to archive

	// Cgo directives
	CgoCFLAGS    []string // cgo: flags for C compiler
	CgoCPPFLAGS  []string // cgo: flags for C preprocessor
	CgoCXXFLAGS  []string // cgo: flags for C++ compiler
	CgoLDFLAGS   []string // cgo: flags for linker
	CgoPkgConfig []string // cgo: pkg-config names

	// Dependency information
	Imports []string // import paths used by this package
	Deps    []string // all (recursively) imported dependencies

	// Error information
	Incomplete bool            // this package or a dependency has an error
	Error      *PackageError   // error loading package
	DepsErrors []*PackageError // errors loading dependencies

	TestGoFiles  []string // _test.go files in package
	TestImports  []string // imports from TestGoFiles
	XTestGoFiles []string // _test.go files outside package
	XTestImports []string // imports from XTestGoFiles
}

// GetPackage is a convience call to look up a single package
func GetPackage(name string) (Package, error) {
	pkgs, err := Packages(name)
	if err != nil {
		return Package{}, err
	}
	if len(pkgs) == 0 {
		return Package{}, fmt.Errorf("package %q not found", name)
	}
	return pkgs[0], nil
}

// Packages is a wrapper around `go list -e -json package...`
//  golang doesn't expose this in a API
//  inpsired by github.com/tools/godep which also doesn't expose this
//  as a library
func Packages(name ...string) ([]Package, error) {
	if len(name) == 0 {
		return nil, nil
	}
	args := []string{"list", "-e", "-json"}
	args = append(args, name...)
	cmd := exec.Command("go", args...)
	r, err := cmd.StdoutPipe()
	if err != nil {
		return nil, err
	}
	cmd.Stderr = os.Stderr
	err = cmd.Start()
	if err != nil {
		return nil, err
	}
	out := make([]Package, 0, 100)
	d := json.NewDecoder(r)
	for {
		info := Package{}
		err = d.Decode(&info)
		if err == io.EOF {
			break
		}
		if err != nil {
			// should never happen
			return nil, err
		}
		out = append(out, info)
	}
	err = cmd.Wait()
	if err != nil {
		return nil, err
	}
	return out, nil
}

// Std calls `go list std` to return a list of standard packages
//  This functionality is not exported programmatically.
func Std() ([]string, error) {
	cmd := exec.Command("go", "list", "std")
	cmd.Stderr = os.Stderr
	stdout, err := cmd.StdoutPipe()
	if err != nil {
		return nil, err
	}
	err = cmd.Start()
	if err != nil {
		return nil, err
	}
	// there are about 148 in go1.5
	std := make([]string, 0, 200)

	scanner := bufio.NewScanner(stdout)
	for scanner.Scan() {
		line := string(scanner.Text())
		std = append(std, line)
	}
	if err := scanner.Err(); err != nil {
		return nil, err
	}
	if err := cmd.Wait(); err != nil {
		return nil, err
	}
	return std, nil
}

// Deps list all depedencies for the given
// list of package paths names returned in sorted order, or error
//
func Deps(name ...string) ([]string, error) {
	if len(name) == 0 {
		return nil, nil
	}
	args := []string{"list", "-f", `{{  join .Deps "\n"}}`}
	args = append(args, name...)
	//	log.Printf("CMD: %v", args)
	cmd := exec.Command("go", args...)
	cmd.Stderr = os.Stderr
	stdout, err := cmd.StdoutPipe()
	if err != nil {
		return nil, err
	}
	err = cmd.Start()
	if err != nil {
		return nil, err
	}
	uniq := make(map[string]bool)
	scanner := bufio.NewScanner(stdout)
	for scanner.Scan() {
		uniq[scanner.Text()] = true
	}
	if err := scanner.Err(); err != nil {
		return nil, err
	}
	if err := cmd.Wait(); err != nil {
		log.Fatalf("GoListDeps Wait failed: %s", err)
	}
	paths := make([]string, 0, len(uniq))
	for k := range uniq {
		paths = append(paths, k)
	}
	sort.Strings(paths)
	return paths, nil
}

// TemplateFuncMap recreates the template environment provided in 'go list'
func TemplateFuncMap() template.FuncMap {
	return template.FuncMap{
		"join":    strings.Join,
		"context": NewContext,
	}
}

const contextTemplate = `{{ with context }}{{ .GOARCH }}
{{ .GOOS }}
{{ .GOROOT }}
{{ .GOPATH }}
{{ .CgoEnabled }}
{{ .UseAllFiles }}
{{ .Compiler }}
{{ join .BuildTags "," }}
{{ join .ReleaseTags "," }}
{{ .InstallSuffix }}{{ end }}`

// NewContext generates a context object
func NewContext() (*Context, error) {
	c := Context{}
	cmd := exec.Command("go", "list", "-f", contextTemplate)
	outbytes, err := cmd.Output()
	if err != nil {
		return nil, err
	}
	lines := strings.Split(string(outbytes), "\n")
	if len(lines) != 10 {
		return nil, fmt.Errorf("expected 10 outlines from golist, got %d with %q", len(lines), string(outbytes))
	}
	c.GOARCH = lines[0]
	c.GOOS = lines[1]
	c.GOROOT = lines[2]
	c.GOPATH = lines[3]
	if lines[4] == "true" {
		c.CgoEnabled = true
	}
	if lines[5] == "true" {
		c.UseAllFiles = true
	}
	c.Compiler = lines[6]
	c.BuildTags = strings.Split(lines[7], ",")
	c.ReleaseTags = strings.Split(lines[8], ",")
	c.InstallSuffix = lines[9]

	return &c, nil
}
<commit_msg>spelling<commit_after>package golist

import (
	"bufio"
	"encoding/json"
	"fmt"
	"io"
	//	"fmt"
	"log"
	"os"
	"os/exec"
	"sort"
	"strings"
	"text/template"
)

// Context is similar to
// // https://github.com/golang/go/blob/master/src/cmd/go/context.go
type Context struct {
	GOARCH        string   // target architecture
	GOOS          string   // target operating system
	GOROOT        string   // Go root
	GOPATH        string   // Go path
	CgoEnabled    bool     // whether cgo can be used
	UseAllFiles   bool     // use files regardless of +build lines, file names
	Compiler      string   // compiler to assume when computing target paths
	BuildTags     []string // build constraints to match in +build lines
	ReleaseTags   []string // releases the current release is compatible with
	InstallSuffix string   // suffix to use in the name of the install dir
}

// A PackageError describes an error loading information about a package.
type PackageError struct {
	ImportStack []string // shortest path from package named on command line to this one
	Pos         string   // position of error
	Err         string   // the error itself
}

// Package is copy of the Package struct as listed in https://golang.org/src/cmd/go/list.go
//  oddly not exported in golang
type Package struct {
	Dir           string // directory containing package sources
	ImportPath    string // import path of package in dir
	ImportComment string // path in import comment on package statement
	Name          string // package name
	Doc           string // package documentation string
	Target        string // install path
	Shlib         string // the shared library that contains this package (only set when -linkshared)
	Goroot        bool   // is this package in the Go root?
	Standard      bool   // is this package part of the standard Go library?
	Stale         bool   // would 'go install' do anything for this package?
	Root          string // Go root or Go path dir containing this package

	// Source files
	GoFiles        []string // .go source files (excluding CgoFiles, TestGoFiles, XTestGoFiles)
	CgoFiles       []string // .go sources files that import "C"
	IgnoredGoFiles []string // .go sources ignored due to build constraints
	CFiles         []string // .c source files
	CXXFiles       []string // .cc, .cxx and .cpp source files
	MFiles         []string // .m source files
	HFiles         []string // .h, .hh, .hpp and .hxx source files
	SFiles         []string // .s source files
	SwigFiles      []string // .swig files
	SwigCXXFiles   []string // .swigcxx files
	SysoFiles      []string // .syso object files to add to archive

	// Cgo directives
	CgoCFLAGS    []string // cgo: flags for C compiler
	CgoCPPFLAGS  []string // cgo: flags for C preprocessor
	CgoCXXFLAGS  []string // cgo: flags for C++ compiler
	CgoLDFLAGS   []string // cgo: flags for linker
	CgoPkgConfig []string // cgo: pkg-config names

	// Dependency information
	Imports []string // import paths used by this package
	Deps    []string // all (recursively) imported dependencies

	// Error information
	Incomplete bool            // this package or a dependency has an error
	Error      *PackageError   // error loading package
	DepsErrors []*PackageError // errors loading dependencies

	TestGoFiles  []string // _test.go files in package
	TestImports  []string // imports from TestGoFiles
	XTestGoFiles []string // _test.go files outside package
	XTestImports []string // imports from XTestGoFiles
}

// GetPackage is a convience call to look up a single package
func GetPackage(name string) (Package, error) {
	pkgs, err := Packages(name)
	if err != nil {
		return Package{}, err
	}
	if len(pkgs) == 0 {
		return Package{}, fmt.Errorf("package %q not found", name)
	}
	return pkgs[0], nil
}

// Packages is a wrapper around `go list -e -json package...`
//  golang doesn't expose this in a API
//  inpsired by github.com/tools/godep which also doesn't expose this
//  as a library
func Packages(name ...string) ([]Package, error) {
	if len(name) == 0 {
		return nil, nil
	}
	args := []string{"list", "-e", "-json"}
	args = append(args, name...)
	cmd := exec.Command("go", args...)
	r, err := cmd.StdoutPipe()
	if err != nil {
		return nil, err
	}
	cmd.Stderr = os.Stderr
	err = cmd.Start()
	if err != nil {
		return nil, err
	}
	out := make([]Package, 0, 100)
	d := json.NewDecoder(r)
	for {
		info := Package{}
		err = d.Decode(&info)
		if err == io.EOF {
			break
		}
		if err != nil {
			// should never happen
			return nil, err
		}
		out = append(out, info)
	}
	err = cmd.Wait()
	if err != nil {
		return nil, err
	}
	return out, nil
}

// Std calls `go list std` to return a list of standard packages
//  This functionality is not exported programmatically.
func Std() ([]string, error) {
	cmd := exec.Command("go", "list", "std")
	cmd.Stderr = os.Stderr
	stdout, err := cmd.StdoutPipe()
	if err != nil {
		return nil, err
	}
	err = cmd.Start()
	if err != nil {
		return nil, err
	}
	// there are about 148 in go1.5
	std := make([]string, 0, 200)

	scanner := bufio.NewScanner(stdout)
	for scanner.Scan() {
		line := string(scanner.Text())
		std = append(std, line)
	}
	if err := scanner.Err(); err != nil {
		return nil, err
	}
	if err := cmd.Wait(); err != nil {
		return nil, err
	}
	return std, nil
}

// Deps list all dependencies for the given
// list of package paths names returned in sorted order, or error
//
func Deps(name ...string) ([]string, error) {
	if len(name) == 0 {
		return nil, nil
	}
	args := []string{"list", "-f", `{{  join .Deps "\n"}}`}
	args = append(args, name...)
	//	log.Printf("CMD: %v", args)
	cmd := exec.Command("go", args...)
	cmd.Stderr = os.Stderr
	stdout, err := cmd.StdoutPipe()
	if err != nil {
		return nil, err
	}
	err = cmd.Start()
	if err != nil {
		return nil, err
	}
	uniq := make(map[string]bool)
	scanner := bufio.NewScanner(stdout)
	for scanner.Scan() {
		uniq[scanner.Text()] = true
	}
	if err := scanner.Err(); err != nil {
		return nil, err
	}
	if err := cmd.Wait(); err != nil {
		log.Fatalf("GoListDeps Wait failed: %s", err)
	}
	paths := make([]string, 0, len(uniq))
	for k := range uniq {
		paths = append(paths, k)
	}
	sort.Strings(paths)
	return paths, nil
}

// TemplateFuncMap recreates the template environment provided in 'go list'
func TemplateFuncMap() template.FuncMap {
	return template.FuncMap{
		"join":    strings.Join,
		"context": NewContext,
	}
}

const contextTemplate = `{{ with context }}{{ .GOARCH }}
{{ .GOOS }}
{{ .GOROOT }}
{{ .GOPATH }}
{{ .CgoEnabled }}
{{ .UseAllFiles }}
{{ .Compiler }}
{{ join .BuildTags "," }}
{{ join .ReleaseTags "," }}
{{ .InstallSuffix }}{{ end }}`

// NewContext generates a context object
func NewContext() (*Context, error) {
	c := Context{}
	cmd := exec.Command("go", "list", "-f", contextTemplate)
	outbytes, err := cmd.Output()
	if err != nil {
		return nil, err
	}
	lines := strings.Split(string(outbytes), "\n")
	if len(lines) != 10 {
		return nil, fmt.Errorf("expected 10 outlines from golist, got %d with %q", len(lines), string(outbytes))
	}
	c.GOARCH = lines[0]
	c.GOOS = lines[1]
	c.GOROOT = lines[2]
	c.GOPATH = lines[3]
	if lines[4] == "true" {
		c.CgoEnabled = true
	}
	if lines[5] == "true" {
		c.UseAllFiles = true
	}
	c.Compiler = lines[6]
	c.BuildTags = strings.Split(lines[7], ",")
	c.ReleaseTags = strings.Split(lines[8], ",")
	c.InstallSuffix = lines[9]

	return &c, nil
}
<|endoftext|>


<commit_before>// Copyright 2011 Rodrigo Moraes. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

/*
Package gorilla/mux implements a request router and dispatcher.

The name mux stands for "HTTP request multiplexer". Like the standard
http.ServeMux, mux.Router matches incoming requests against a list of
registered routes and calls a handler for the route that matches the URL
or other conditions. The main features are:

* URL hosts and paths can be defined using named variables with an optional
regexp.

* Registered URLs can be built, or "reversed", which helps maintaining
references to resources.

* Requests can also be matched based on HTTP methods, URL schemes, header and
query values or using custom matchers.

* Routes can be nested, so that they are only tested if the parent route
matches. This allows defining groups of routes that share common conditions
like a host, a path prefix or other repeated attributes. As a bonus, this
optimizes request matching.

* It is compatible with http.ServeMux: it can register handlers that implement
the http.Handler interface or the signature accepted by http.HandleFunc().

The most basic example is to register a couple of URL paths and handlers:

	mux.HandleFunc("/", HomeHandler)
	mux.HandleFunc("/products", ProductsHandler)
	mux.HandleFunc("/articles", ArticlesHandler)

Here we register three routes mapping URL paths to handlers. This is
equivalent to how http.HandleFunc() works: if an incoming request URL matches
one of the paths, the corresponding handler is called passing
(http.ResponseWriter, *http.Request) as parameters.

Paths can have variables. They are defined using the notation {name} or
{name:pattern}. If a regular expression pattern is not defined, the variable
will be anything until the next slash. For example:

	mux.HandleFunc("/products/{key}", ProductHandler)
	mux.HandleFunc("/articles/{category}/", ArticlesCategoryHandler)
	mux.HandleFunc("/articles/{category}/{id:[0-9]+}", ArticleHandler)

The names are used to create a map of route variables which can be retrieved
calling mux.Vars():

	vars := mux.Vars(request)
	category := vars["category"]

And this is all you need to know about the basic usage. More advanced options
are explained below.

Routes can also be restricted to a domain or subdomain. Just define a host
pattern to be matched. They can also have variables:

	// Only matches if domain is "www.domain.com".
	mux.HandleFunc("/products", ProductsHandler).Host("www.domain.com")
	// Matches a dynamic subdomain.
	mux.HandleFunc("/products", ProductsHandler).
		Host("{subdomain:[a-z]+}.domain.com")

There are several other matchers that can be added. To match HTTP methods:

	mux.HandleFunc("/products", ProductsHandler).Methods("GET", "POST")

...or to match a given URL scheme:

	mux.HandleFunc("/products", ProductsHandler).Schemes("https")

...or to match specific header values:

	mux.HandleFunc("/products", ProductsHandler).
		Headers("X-Requested-With", "XMLHttpRequest")

...or to match specific URL query values:

	mux.HandleFunc("/products", ProductsHandler).Queries("key", "value")

...or to use a custom matcher function:

	mux.HandleFunc("/products", ProductsHandler).Matcher(MatcherFunc)

...and finally, it is possible to combine several matchers in a single route:

	mux.HandleFunc("/products", ProductsHandler).
		Host("www.domain.com").
		Methods("GET").Schemes("http")

Setting the same matching conditions again and again can be boring, so we have
a way to group several routes that share the same requirements.
We call it "subrouting".

For example, let's say we have several URLs that should only match when the
host is "www.domain.com". We create a route for that host, then add a
"subrouter" to that route:

	subrouter := mux.Host("www.domain.com").NewRouter()

Then register routes for the host subrouter:

	subrouter.HandleFunc("/products/", ProductsHandler)
	subrouter.HandleFunc("/products/{key}", ProductHandler)
	subrouter.HandleFunc("/articles/{category}/{id:[0-9]+}"), ArticleHandler)

The three URL paths we registered above will only be tested if the domain is
"www.domain.com", because the subrouter is tested first. This is not
only convenient, but also optimizes request matching. You can create
subrouters combining any attribute matchers accepted by a route.

Now let's see how to build registered URLs.

Routes can be named. All routes that define a name can have their URLs built,
or "reversed". We define a name calling Name() on a route. For example:

	mux.HandleFunc("/articles/{category}/{id:[0-9]+}", ArticleHandler).
		Name("article")

Named routes are available in the NamedRoutes field from a router. To build
a URL, get the route and call the URL() method, passing a sequence of
key/value pairs for the route variables. For the previous route, we would do:

	url := mux.NamedRoutes["article"].URL("category", "technology", "id", "42")

...and the result will be an http.URL with the following path:

	"/articles/technology/42"

This also works for host variables:

	mux.Host("{subdomain}.domain.com").
		HandleFunc("/articles/{category}/{id:[0-9]+}", ArticleHandler).
		Name("article")

	// url.String() will be "http://news.domain.com/articles/technology/42"
	url := mux.NamedRoutes["article"].URL("subdomain", "news",
										  "category", "technology",
										  "id", "42")

All variable names defined in the route are required, and their values must
conform to the corresponding patterns, if any.

There's also a way to build only the URL host or path for a route:
use the methods URLHost() or URLPath() instead. For the previous route,
we would do:

	// "http://news.domain.com/"
	host := mux.NamedRoutes["article"].URLHost("subdomain", "news").String()

	// "/articles/technology/42"
	path := mux.NamedRoutes["article"].URLPath("category", "technology",
											   "id", "42").String()
*/
package mux
<commit_msg>http.URL ---> url.URL in doc.<commit_after>// Copyright 2011 Rodrigo Moraes. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

/*
Package gorilla/mux implements a request router and dispatcher.

The name mux stands for "HTTP request multiplexer". Like the standard
http.ServeMux, mux.Router matches incoming requests against a list of
registered routes and calls a handler for the route that matches the URL
or other conditions. The main features are:

* URL hosts and paths can be defined using named variables with an optional
regexp.

* Registered URLs can be built, or "reversed", which helps maintaining
references to resources.

* Requests can also be matched based on HTTP methods, URL schemes, header and
query values or using custom matchers.

* Routes can be nested, so that they are only tested if the parent route
matches. This allows defining groups of routes that share common conditions
like a host, a path prefix or other repeated attributes. As a bonus, this
optimizes request matching.

* It is compatible with http.ServeMux: it can register handlers that implement
the http.Handler interface or the signature accepted by http.HandleFunc().

The most basic example is to register a couple of URL paths and handlers:

	mux.HandleFunc("/", HomeHandler)
	mux.HandleFunc("/products", ProductsHandler)
	mux.HandleFunc("/articles", ArticlesHandler)

Here we register three routes mapping URL paths to handlers. This is
equivalent to how http.HandleFunc() works: if an incoming request URL matches
one of the paths, the corresponding handler is called passing
(http.ResponseWriter, *http.Request) as parameters.

Paths can have variables. They are defined using the notation {name} or
{name:pattern}. If a regular expression pattern is not defined, the variable
will be anything until the next slash. For example:

	mux.HandleFunc("/products/{key}", ProductHandler)
	mux.HandleFunc("/articles/{category}/", ArticlesCategoryHandler)
	mux.HandleFunc("/articles/{category}/{id:[0-9]+}", ArticleHandler)

The names are used to create a map of route variables which can be retrieved
calling mux.Vars():

	vars := mux.Vars(request)
	category := vars["category"]

And this is all you need to know about the basic usage. More advanced options
are explained below.

Routes can also be restricted to a domain or subdomain. Just define a host
pattern to be matched. They can also have variables:

	// Only matches if domain is "www.domain.com".
	mux.HandleFunc("/products", ProductsHandler).Host("www.domain.com")
	// Matches a dynamic subdomain.
	mux.HandleFunc("/products", ProductsHandler).
		Host("{subdomain:[a-z]+}.domain.com")

There are several other matchers that can be added. To match HTTP methods:

	mux.HandleFunc("/products", ProductsHandler).Methods("GET", "POST")

...or to match a given URL scheme:

	mux.HandleFunc("/products", ProductsHandler).Schemes("https")

...or to match specific header values:

	mux.HandleFunc("/products", ProductsHandler).
		Headers("X-Requested-With", "XMLHttpRequest")

...or to match specific URL query values:

	mux.HandleFunc("/products", ProductsHandler).Queries("key", "value")

...or to use a custom matcher function:

	mux.HandleFunc("/products", ProductsHandler).Matcher(MatcherFunc)

...and finally, it is possible to combine several matchers in a single route:

	mux.HandleFunc("/products", ProductsHandler).
		Host("www.domain.com").
		Methods("GET").Schemes("http")

Setting the same matching conditions again and again can be boring, so we have
a way to group several routes that share the same requirements.
We call it "subrouting".

For example, let's say we have several URLs that should only match when the
host is "www.domain.com". We create a route for that host, then add a
"subrouter" to that route:

	subrouter := mux.Host("www.domain.com").NewRouter()

Then register routes for the host subrouter:

	subrouter.HandleFunc("/products/", ProductsHandler)
	subrouter.HandleFunc("/products/{key}", ProductHandler)
	subrouter.HandleFunc("/articles/{category}/{id:[0-9]+}"), ArticleHandler)

The three URL paths we registered above will only be tested if the domain is
"www.domain.com", because the subrouter is tested first. This is not
only convenient, but also optimizes request matching. You can create
subrouters combining any attribute matchers accepted by a route.

Now let's see how to build registered URLs.

Routes can be named. All routes that define a name can have their URLs built,
or "reversed". We define a name calling Name() on a route. For example:

	mux.HandleFunc("/articles/{category}/{id:[0-9]+}", ArticleHandler).
		Name("article")

Named routes are available in the NamedRoutes field from a router. To build
a URL, get the route and call the URL() method, passing a sequence of
key/value pairs for the route variables. For the previous route, we would do:

	url := mux.NamedRoutes["article"].URL("category", "technology", "id", "42")

...and the result will be a url.URL with the following path:

	"/articles/technology/42"

This also works for host variables:

	mux.Host("{subdomain}.domain.com").
		HandleFunc("/articles/{category}/{id:[0-9]+}", ArticleHandler).
		Name("article")

	// url.String() will be "http://news.domain.com/articles/technology/42"
	url := mux.NamedRoutes["article"].URL("subdomain", "news",
										  "category", "technology",
										  "id", "42")

All variable names defined in the route are required, and their values must
conform to the corresponding patterns, if any.

There's also a way to build only the URL host or path for a route:
use the methods URLHost() or URLPath() instead. For the previous route,
we would do:

	// "http://news.domain.com/"
	host := mux.NamedRoutes["article"].URLHost("subdomain", "news").String()

	// "/articles/technology/42"
	path := mux.NamedRoutes["article"].URLPath("category", "technology",
											   "id", "42").String()
*/
package mux
<|endoftext|>


<commit_before>// Copyright 2017 Google Inc. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package s2

import (
	"sort"
)

// dimension defines the types of geometry dimensions that a Shape supports.
type dimension int

const (
	pointGeometry dimension = iota
	polylineGeometry
	polygonGeometry
)

// Edge represents a geodesic edge consisting of two vertices. Zero-length edges are
// allowed, and can be used to represent points.
type Edge struct {
	V0, V1 Point
}

// Cmp compares the two edges using the underlying Points Cmp method and returns
//
//   -1 if e <  other
//    0 if e == other
//   +1 if e >  other
//
// The two edges are compared by first vertex, and then by the second vertex.
func (e Edge) Cmp(other Edge) int {
	if v0cmp := e.V0.Cmp(other.V0.Vector); v0cmp != 0 {
		return v0cmp
	}
	return e.V1.Cmp(other.V1.Vector)
}

// sortEdges sorts the slice of Edges in place.
func sortEdges(e []Edge) {
	sort.Sort(edges(e))
}

// edges implements the Sort interface for slices of Edge.
type edges []Edge

func (e edges) Len() int           { return len(e) }
func (e edges) Swap(i, j int)      { e[i], e[j] = e[j], e[i] }
func (e edges) Less(i, j int) bool { return e[i].Cmp(e[j]) == -1 }

// Chain represents a range of edge IDs corresponding to a chain of connected
// edges, specified as a (start, length) pair. The chain is defined to consist of
// edge IDs {start, start + 1, ..., start + length - 1}.
type Chain struct {
	Start, Length int
}

// ChainPosition represents the position of an edge within a given edge chain,
// specified as a (chainID, offset) pair. Chains are numbered sequentially
// starting from zero, and offsets are measured from the start of each chain.
type ChainPosition struct {
	ChainID, Offset int
}

// A ReferencePoint consists of a point and a boolean indicating whether the point
// is contained by a particular shape.
type ReferencePoint struct {
	Point     Point
	Contained bool
}

// OriginReferencePoint returns a ReferencePoint with the given value for
// contained and the origin point. It should be used when all points or no
// points are contained.
func OriginReferencePoint(contained bool) ReferencePoint {
	return ReferencePoint{Point: OriginPoint(), Contained: contained}
}

// Shape represents polygonal geometry in a flexible way. It is organized as a
// collection of edges that optionally defines an interior. All geometry
// represented by a given Shape must have the same dimension, which means that
// an Shape can represent either a set of points, a set of polylines, or a set
// of polygons.
//
// Shape is defined as an interface in order to give clients control over the
// underlying data representation. Sometimes an Shape does not have any data of
// its own, but instead wraps some other type.
//
// Shape operations are typically defined on a ShapeIndex rather than
// individual shapes. An ShapeIndex is simply a collection of Shapes,
// possibly of different dimensions (e.g. 10 points and 3 polygons), organized
// into a data structure for efficient edge access.
//
// The edges of a Shape are indexed by a contiguous range of edge IDs
// starting at 0. The edges are further subdivided into chains, where each
// chain consists of a sequence of edges connected end-to-end (a polyline).
// For example, a Shape representing two polylines AB and CDE would have
// three edges (AB, CD, DE) grouped into two chains: (AB) and (CD, DE).
// Similarly, an Shape representing 5 points would have 5 chains consisting
// of one edge each.
//
// Shape has methods that allow edges to be accessed either using the global
// numbering (edge ID) or within a particular chain. The global numbering is
// sufficient for most purposes, but the chain representation is useful for
// certain algorithms such as intersection (see BooleanOperation).
type Shape interface {
	// NumEdges returns the number of edges in this shape.
	NumEdges() int

	// Edge returns the edge for the given edge index.
	Edge(i int) Edge

	// HasInterior reports whether this shape has an interior.
	HasInterior() bool

	// ReferencePoint returns an arbitrary reference point for the shape. (The
	// containment boolean value must be false for shapes that do not have an interior.)
	//
	// This reference point may then be used to compute the containment of other
	// points by counting edge crossings.
	ReferencePoint() ReferencePoint

	// NumChains reports the number of contiguous edge chains in the shape.
	// For example, a shape whose edges are [AB, BC, CD, AE, EF] would consist
	// of two chains (AB,BC,CD and AE,EF). Every chain is assigned a chain Id
	// numbered sequentially starting from zero.
	//
	// Note that it is always acceptable to implement this method by returning
	// NumEdges, i.e. every chain consists of a single edge, but this may
	// reduce the efficiency of some algorithms.
	NumChains() int

	// Chain returns the range of edge IDs corresponding to the given edge chain.
	// Edge chains must form contiguous, non-overlapping ranges that cover
	// the entire range of edge IDs. This is spelled out more formally below:
	//
	//  0 <= i < NumChains()
	//  Chain(i).length > 0, for all i
	//  Chain(0).start == 0
	//  Chain(i).start + Chain(i).length == Chain(i+1).start, for i < NumChains()-1
	//  Chain(i).start + Chain(i).length == NumEdges(), for i == NumChains()-1
	Chain(chainID int) Chain

	// ChainEdgeReturns the edge at offset "offset" within edge chain "chainID".
	// Equivalent to "shape.Edge(shape.Chain(chainID).start + offset)"
	// but more efficient.
	ChainEdge(chainID, offset int) Edge

	// ChainPosition finds the chain containing the given edge, and returns the
	// position of that edge as a ChainPosition(chainID, offset) pair.
	//
	//  shape.Chain(pos.chainID).start + pos.offset == edgeID
	//  shape.Chain(pos.chainID+1).start > edgeID
	//
	// where pos == shape.ChainPosition(edgeID).
	ChainPosition(edgeID int) ChainPosition

	// dimension returns the dimension of the geometry represented by this shape.
	//
	//  pointGeometry: Each point is represented as a degenerate edge.
	//
	//  polylineGeometry:  Polyline edges may be degenerate. A shape may
	//      represent any number of polylines. Polylines edges may intersect.
	//
	//  polygonGeometry:  Edges should be oriented such that the polygon
	//      interior is always on the left. In theory the edges may be returned
	//      in any order, but typically the edges are organized as a collection
	//      of edge chains where each chain represents one polygon loop.
	//      Polygons may have degeneracies (e.g., degenerate edges or sibling
	//      pairs consisting of an edge and its corresponding reversed edge).
	//
	// Note that this method allows degenerate geometry of different dimensions
	// to be distinguished, e.g. it allows a point to be distinguished from a
	// polyline or polygon that has been simplified to a single point.
	dimension() dimension
}

// A minimal check for types that should satisfy the Shape interface.
var (
	_ Shape = &Loop{}
	_ Shape = &Polygon{}
	_ Shape = &Polyline{}
)
<commit_msg>s2: Add ShapeEdgeID and ShapeEdge types.<commit_after>// Copyright 2017 Google Inc. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package s2

import (
	"sort"
)

// dimension defines the types of geometry dimensions that a Shape supports.
type dimension int

const (
	pointGeometry dimension = iota
	polylineGeometry
	polygonGeometry
)

// Edge represents a geodesic edge consisting of two vertices. Zero-length edges are
// allowed, and can be used to represent points.
type Edge struct {
	V0, V1 Point
}

// Cmp compares the two edges using the underlying Points Cmp method and returns
//
//   -1 if e <  other
//    0 if e == other
//   +1 if e >  other
//
// The two edges are compared by first vertex, and then by the second vertex.
func (e Edge) Cmp(other Edge) int {
	if v0cmp := e.V0.Cmp(other.V0.Vector); v0cmp != 0 {
		return v0cmp
	}
	return e.V1.Cmp(other.V1.Vector)
}

// sortEdges sorts the slice of Edges in place.
func sortEdges(e []Edge) {
	sort.Sort(edges(e))
}

// edges implements the Sort interface for slices of Edge.
type edges []Edge

func (e edges) Len() int           { return len(e) }
func (e edges) Swap(i, j int)      { e[i], e[j] = e[j], e[i] }
func (e edges) Less(i, j int) bool { return e[i].Cmp(e[j]) == -1 }

// ShapeEdgeID is a unique identifier for an Edge within an ShapeIndex,
// consisting of a (shapeID, edgeID) pair.
type ShapeEdgeID struct {
	ShapeID int32
	EdgeID  int32
}

// Cmp compares the two ShapeEdgeIDs and returns
//
//   -1 if s <  other
//    0 if s == other
//   +1 if s >  other
//
// The two are compared first by shape id and then by edge id.
func (s ShapeEdgeID) Cmp(other ShapeEdgeID) int {
	switch {
	case s.ShapeID < other.ShapeID:
		return -1
	case s.ShapeID > other.ShapeID:
		return 1
	}
	switch {
	case s.EdgeID < other.EdgeID:
		return -1
	case s.EdgeID > other.EdgeID:
		return 1
	}
	return 0
}

// ShapeEdge represents a ShapeEdgeID with the two endpoints of that Edge.
type ShapeEdge struct {
	ID   ShapeEdgeID
	Edge Edge
}

// Chain represents a range of edge IDs corresponding to a chain of connected
// edges, specified as a (start, length) pair. The chain is defined to consist of
// edge IDs {start, start + 1, ..., start + length - 1}.
type Chain struct {
	Start, Length int
}

// ChainPosition represents the position of an edge within a given edge chain,
// specified as a (chainID, offset) pair. Chains are numbered sequentially
// starting from zero, and offsets are measured from the start of each chain.
type ChainPosition struct {
	ChainID, Offset int
}

// A ReferencePoint consists of a point and a boolean indicating whether the point
// is contained by a particular shape.
type ReferencePoint struct {
	Point     Point
	Contained bool
}

// OriginReferencePoint returns a ReferencePoint with the given value for
// contained and the origin point. It should be used when all points or no
// points are contained.
func OriginReferencePoint(contained bool) ReferencePoint {
	return ReferencePoint{Point: OriginPoint(), Contained: contained}
}

// Shape represents polygonal geometry in a flexible way. It is organized as a
// collection of edges that optionally defines an interior. All geometry
// represented by a given Shape must have the same dimension, which means that
// an Shape can represent either a set of points, a set of polylines, or a set
// of polygons.
//
// Shape is defined as an interface in order to give clients control over the
// underlying data representation. Sometimes an Shape does not have any data of
// its own, but instead wraps some other type.
//
// Shape operations are typically defined on a ShapeIndex rather than
// individual shapes. An ShapeIndex is simply a collection of Shapes,
// possibly of different dimensions (e.g. 10 points and 3 polygons), organized
// into a data structure for efficient edge access.
//
// The edges of a Shape are indexed by a contiguous range of edge IDs
// starting at 0. The edges are further subdivided into chains, where each
// chain consists of a sequence of edges connected end-to-end (a polyline).
// For example, a Shape representing two polylines AB and CDE would have
// three edges (AB, CD, DE) grouped into two chains: (AB) and (CD, DE).
// Similarly, an Shape representing 5 points would have 5 chains consisting
// of one edge each.
//
// Shape has methods that allow edges to be accessed either using the global
// numbering (edge ID) or within a particular chain. The global numbering is
// sufficient for most purposes, but the chain representation is useful for
// certain algorithms such as intersection (see BooleanOperation).
type Shape interface {
	// NumEdges returns the number of edges in this shape.
	NumEdges() int

	// Edge returns the edge for the given edge index.
	Edge(i int) Edge

	// HasInterior reports whether this shape has an interior.
	HasInterior() bool

	// ReferencePoint returns an arbitrary reference point for the shape. (The
	// containment boolean value must be false for shapes that do not have an interior.)
	//
	// This reference point may then be used to compute the containment of other
	// points by counting edge crossings.
	ReferencePoint() ReferencePoint

	// NumChains reports the number of contiguous edge chains in the shape.
	// For example, a shape whose edges are [AB, BC, CD, AE, EF] would consist
	// of two chains (AB,BC,CD and AE,EF). Every chain is assigned a chain Id
	// numbered sequentially starting from zero.
	//
	// Note that it is always acceptable to implement this method by returning
	// NumEdges, i.e. every chain consists of a single edge, but this may
	// reduce the efficiency of some algorithms.
	NumChains() int

	// Chain returns the range of edge IDs corresponding to the given edge chain.
	// Edge chains must form contiguous, non-overlapping ranges that cover
	// the entire range of edge IDs. This is spelled out more formally below:
	//
	//  0 <= i < NumChains()
	//  Chain(i).length > 0, for all i
	//  Chain(0).start == 0
	//  Chain(i).start + Chain(i).length == Chain(i+1).start, for i < NumChains()-1
	//  Chain(i).start + Chain(i).length == NumEdges(), for i == NumChains()-1
	Chain(chainID int) Chain

	// ChainEdgeReturns the edge at offset "offset" within edge chain "chainID".
	// Equivalent to "shape.Edge(shape.Chain(chainID).start + offset)"
	// but more efficient.
	ChainEdge(chainID, offset int) Edge

	// ChainPosition finds the chain containing the given edge, and returns the
	// position of that edge as a ChainPosition(chainID, offset) pair.
	//
	//  shape.Chain(pos.chainID).start + pos.offset == edgeID
	//  shape.Chain(pos.chainID+1).start > edgeID
	//
	// where pos == shape.ChainPosition(edgeID).
	ChainPosition(edgeID int) ChainPosition

	// dimension returns the dimension of the geometry represented by this shape.
	//
	//  pointGeometry: Each point is represented as a degenerate edge.
	//
	//  polylineGeometry:  Polyline edges may be degenerate. A shape may
	//      represent any number of polylines. Polylines edges may intersect.
	//
	//  polygonGeometry:  Edges should be oriented such that the polygon
	//      interior is always on the left. In theory the edges may be returned
	//      in any order, but typically the edges are organized as a collection
	//      of edge chains where each chain represents one polygon loop.
	//      Polygons may have degeneracies (e.g., degenerate edges or sibling
	//      pairs consisting of an edge and its corresponding reversed edge).
	//
	// Note that this method allows degenerate geometry of different dimensions
	// to be distinguished, e.g. it allows a point to be distinguished from a
	// polyline or polygon that has been simplified to a single point.
	dimension() dimension
}

// A minimal check for types that should satisfy the Shape interface.
var (
	_ Shape = &Loop{}
	_ Shape = &Polygon{}
	_ Shape = &Polyline{}
)
<|endoftext|>


<commit_before>package main

import (
	"flag"
	"fmt"
	"log"
	"os"
	"strconv"
	"syscall"

	"github.com/crowdmob/goamz/aws"
	"github.com/crowdmob/goamz/s3"
)

var logger *log.Logger

// flags
var awsKey *string
var awsSecret *string
var s3Region *string
var start *int
var end *int
var width *int
var bucket *string
var prefix *string

func main() {
	if *bucket == "" {
		logger.Println("Bucket not specified")
		syscall.Exit(1)
	}

	auth := new(aws.Auth)
	auth.AccessKey = *awsKey
	auth.SecretKey = *awsSecret
	s3c := s3.New(*auth, aws.Regions[*s3Region])
	s3bucket := s3c.Bucket(*bucket)
	logger.Println(s3bucket)

	// making i with leading zeros with this format
	format := "%0" + strconv.Itoa(*width) + "d"
	for i := *start; i <= *end; i += 1 {
		suffix := fmt.Sprintf(format, i)
		key := *prefix + suffix
		logger.Printf("Deleting S3 key: %s", key)
		err := s3bucket.Del(key)
		if err != nil {
			logger.Printf("Got error deleting key: %s", err)
		}
	}
}

func init() {
	logger = log.New(os.Stdout, "", 0)

	awsKey = flag.String("awsKey", os.Getenv("AWS_ACCESS_KEY_ID"), "AWS Key. Defaults to env var AWS_ACCESS_KEY_ID")
	awsSecret = flag.String("awsSecret", os.Getenv("AWS_SECRET_KEY"), "AWS Secret. Defaults to env var AWS_SECRET_KEY")
	s3Region = flag.String("s3Region", "us-east-1", "AWS S3 region")
	start = flag.Int("start", 0, "Starting number")
	end = flag.Int("end", 0, "Ending number (inclusive)")
	width = flag.Int("width", 6, "Key number width (ex. when width = 6, 1 = 000001)")
	bucket = flag.String("bucket", "", "Bucket")
	prefix = flag.String("prefix", "/", "Key prefix")
	flag.Parse()
}

<commit_msg>used address operator for flag parsing to eliminate over-use of pointers in the rest of the code<commit_after>package main

import (
	"flag"
	"fmt"
	"log"
	"os"
	"strconv"
	"syscall"

	"github.com/crowdmob/goamz/aws"
	"github.com/crowdmob/goamz/s3"
)

var logger *log.Logger

// flags
var awsKey string
var awsSecret string
var s3Region string
var start int
var end int
var width int
var bucket string
var prefix string

func main() {
	if bucket == "" {
		logger.Println("Bucket not specified")
		syscall.Exit(1)
	}

	auth := new(aws.Auth)
	auth.AccessKey = awsKey
	auth.SecretKey = awsSecret
	s3c := s3.New(*auth, aws.Regions[s3Region])
	s3bucket := s3c.Bucket(bucket)
	logger.Println(s3bucket)

	// making i with leading zeros with this format
	format := "%0" + strconv.Itoa(width) + "d"
	for i := start; i <= end; i += 1 {
		suffix := fmt.Sprintf(format, i)
		key := prefix + suffix
		logger.Printf("Deleting S3 key: %s", key)
		err := s3bucket.Del(key)
		if err != nil {
			logger.Printf("Got error deleting key: %s", err)
		}
	}
}

func init() {
	logger = log.New(os.Stdout, "", 0)

	flag.StringVar(&awsKey, "awsKey", os.Getenv("AWS_ACCESS_KEY_ID"), "AWS Key. Defaults to env var AWS_ACCESS_KEY_ID")
	flag.StringVar(&awsSecret, "awsSecret", os.Getenv("AWS_SECRET_KEY"), "AWS Secret. Defaults to env var AWS_SECRET_KEY")
	flag.StringVar(&s3Region, "s3Region", "us-east-1", "AWS S3 region")
	flag.IntVar(&start, "start", 0, "Starting number")
	flag.IntVar(&end, "end", 0, "Ending number (inclusive)")
	flag.IntVar(&width, "width", 6, "Key number width (ex. when width = 6, 1 = 000001)")
	flag.StringVar(&bucket, "bucket", "", "Bucket")
	flag.StringVar(&prefix, "prefix", "/", "Key prefix")
	flag.Parse()
}
<|endoftext|>


<commit_before>package gocql

import "fmt"

// HostFilter interface is used when a host is discovered via server sent events.
type HostFilter interface {
	// Called when a new host is discovered, returning true will cause the host
	// to be added to the pools.
	Accept(host *HostInfo) bool
}

// HostFilterFunc converts a func(host HostInfo) bool into a HostFilter
type HostFilterFunc func(host *HostInfo) bool

func (fn HostFilterFunc) Accept(host *HostInfo) bool {
	return fn(host)
}

// AcceptAllFilter will accept all hosts
func AcceptAllFilter() HostFilter {
	return HostFilterFunc(func(host *HostInfo) bool {
		return true
	})
}

func DenyAllFilter() HostFilter {
	return HostFilterFunc(func(host *HostInfo) bool {
		return false
	})
}

// DataCentreHostFilter filters all hosts such that they are in the same data centre
// as the supplied data centre.
func DataCentreHostFilter(dataCentre string) HostFilter {
	return HostFilterFunc(func(host *HostInfo) bool {
		return host.DataCenter() == dataCentre
	})
}

// WhiteListHostFilter filters incoming hosts by checking that their address is
// in the initial hosts whitelist.
func WhiteListHostFilter(hosts ...string) HostFilter {
	hostInfos, err := addrsToHosts(hosts, 9042)
	if err != nil {
		// dont want to panic here, but rather not break the API
		panic(fmt.Errorf("unable to lookup host info from address: %v", err))
	}

	m := make(map[string]bool, len(hostInfos))
	for _, host := range hostInfos {
		m[string(host.ConnectAddress())] = true
	}

	return HostFilterFunc(func(host *HostInfo) bool {
		return m[string(host.ConnectAddress())]
	})
}
<commit_msg>WhiteListHostFilter: use IP.String() for the filter key (#926)<commit_after>package gocql

import "fmt"

// HostFilter interface is used when a host is discovered via server sent events.
type HostFilter interface {
	// Called when a new host is discovered, returning true will cause the host
	// to be added to the pools.
	Accept(host *HostInfo) bool
}

// HostFilterFunc converts a func(host HostInfo) bool into a HostFilter
type HostFilterFunc func(host *HostInfo) bool

func (fn HostFilterFunc) Accept(host *HostInfo) bool {
	return fn(host)
}

// AcceptAllFilter will accept all hosts
func AcceptAllFilter() HostFilter {
	return HostFilterFunc(func(host *HostInfo) bool {
		return true
	})
}

func DenyAllFilter() HostFilter {
	return HostFilterFunc(func(host *HostInfo) bool {
		return false
	})
}

// DataCentreHostFilter filters all hosts such that they are in the same data centre
// as the supplied data centre.
func DataCentreHostFilter(dataCentre string) HostFilter {
	return HostFilterFunc(func(host *HostInfo) bool {
		return host.DataCenter() == dataCentre
	})
}

// WhiteListHostFilter filters incoming hosts by checking that their address is
// in the initial hosts whitelist.
func WhiteListHostFilter(hosts ...string) HostFilter {
	hostInfos, err := addrsToHosts(hosts, 9042)
	if err != nil {
		// dont want to panic here, but rather not break the API
		panic(fmt.Errorf("unable to lookup host info from address: %v", err))
	}

	m := make(map[string]bool, len(hostInfos))
	for _, host := range hostInfos {
		m[host.ConnectAddress().String()] = true
	}

	return HostFilterFunc(func(host *HostInfo) bool {
		return m[host.ConnectAddress().String()]
	})
}
<|endoftext|>