Migrate to dep (#3972)

* Update makefile to use dep

* Migrate to dep

* Fix some deps

* Try to find a better version for golang.org/x/net

* Try to find a better version for golang.org/x/oauth2
This commit is contained in:
Antoine GIRARD
2018-05-21 14:34:20 +02:00
committed by Lauris BH
parent d7fd9bf7bb
commit 3f3383dc0a
281 changed files with 12024 additions and 32676 deletions

875
Gopkg.lock generated Normal file

File diff suppressed because it is too large Load Diff

106
Gopkg.toml Normal file
View File

@ -0,0 +1,106 @@
ignored = ["google.golang.org/appengine*"]
[prune]
go-tests = true
unused-packages = true
non-go = true
[[constraint]]
branch = "master"
name = "code.gitea.io/git"
[[constraint]]
branch = "master"
name = "code.gitea.io/sdk"
[[constraint]]
revision = "9f005a07e0d31d45e6656d241bb5c0f2efd4bc94"
name = "golang.org/x/crypto"
[[constraint]]
revision = "a646d33e2ee3172a661fc09bca23bb4889a41bc8"
name = "golang.org/x/sys"
[[constraint]]
revision = "2bf8f2a19ec09c670e931282edfe6567f6be21c9"
name = "golang.org/x/text"
[[constraint]]
revision = "f2499483f923065a842d38eb4c7f1927e6fc6e6d"
name = "golang.org/x/net"
[[constraint]]
#version = "v1.0.0"
revision = "33197485abe227dcb254644cf5081c9a3c281669"
name = "github.com/pingcap/tidb"
[[override]]
name = "github.com/go-xorm/xorm"
#version = "0.6.5"
revision = "d4149d1eee0c2c488a74a5863fd9caf13d60fd03"
[[override]]
name = "github.com/gorilla/mux"
revision = "757bef944d0f21880861c2dd9c871ca543023cba"
[[constraint]]
name = "github.com/gorilla/context"
version = "1.1.1"
[[constraint]]
name = "github.com/lafriks/xormstore"
version = "1.0.0"
[[constraint]]
branch = "master"
name = "github.com/lunny/dingtalk_webhook"
[[constraint]]
name = "github.com/markbates/goth"
version = "1.45.5"
[[constraint]]
branch = "master"
name = "github.com/mcuadros/go-version"
[[constraint]]
branch = "master"
name = "github.com/russross/blackfriday"
[[constraint]]
branch = "master"
name = "github.com/tstranex/u2f"
[[constraint]]
name = "gopkg.in/editorconfig/editorconfig-core-go.v1"
version = "1.2.0"
[[constraint]]
branch = "v2"
name = "gopkg.in/gomail.v2"
[[constraint]]
name = "gopkg.in/ini.v1"
version = "1.31.1"
[[constraint]]
name = "gopkg.in/ldap.v2"
version = "2.4.1"
[[constraint]]
name = "gopkg.in/macaron.v1"
version = "1.2.4"
[[constraint]]
name = "gopkg.in/testfixtures.v2"
version = "2.0.0"
[[override]]
name = "github.com/boltdb/bolt"
revision = "ccd680d8c1a0179ac3d68f692b01e1a1589cbfc7"
source = "github.com/go-gitea/bolt"
[[override]]
revision = "c10ba270aa0bf8b8c1c986e103859c67a9103061"
name = "golang.org/x/oauth2"

View File

@ -153,18 +153,22 @@ coverage:
unit-test-coverage:
for PKG in $(PACKAGES); do $(GO) test -tags=sqlite -cover -coverprofile $$GOPATH/src/$$PKG/coverage.out $$PKG || exit 1; done;
.PHONY: test-vendor
test-vendor:
@hash govendor > /dev/null 2>&1; if [ $$? -ne 0 ]; then \
$(GO) get -u github.com/kardianos/govendor; \
.PHONY: vendor
vendor:
@hash dep > /dev/null 2>&1; if [ $$? -ne 0 ]; then \
$(GO) get -u github.com/golang/dep/cmd/dep; \
fi
govendor list +unused | tee "$(TMPDIR)/wc-gitea-unused"
[ $$(cat "$(TMPDIR)/wc-gitea-unused" | wc -l) -eq 0 ] || echo "Warning: /!\\ Some vendor are not used /!\\"
dep ensure -vendor-only
govendor list +outside | tee "$(TMPDIR)/wc-gitea-outside"
[ $$(cat "$(TMPDIR)/wc-gitea-outside" | wc -l) -eq 0 ] || exit 1
govendor status || exit 1
.PHONY: test-vendor
test-vendor: vendor
@diff=$$(git diff vendor/); \
if [ -n "$$diff" ]; then \
echo "Please run 'make vendor' and commit the result:"; \
echo "$${diff}"; \
exit 1; \
fi;
#TODO add dep status -missing when implemented
.PHONY: test-sqlite
test-sqlite: integrations.sqlite.test

View File

@ -1,86 +0,0 @@
# Contribution Guidelines
## Introduction
This document explains how to contribute changes to the Gitea project. It assumes you have followed the [installation instructions](https://github.com/go-gitea/docs/tree/master/en-US/installation). Sensitive security-related issues should be reported to [security@gitea.io](mailto:security@gitea.io).
## Bug reports
Please search the issues on the issue tracker with a variety of keywords to ensure your bug is not already reported.
If unique, [open an issue](https://github.com/go-gitea/gitea/issues/new) and answer the questions so we can understand and reproduce the problematic behavior.
To show us that the issue you are having is in Gitea itself, please write clear, concise instructions so we can reproduce the behavior (even if it seems obvious). The more detailed and specific you are, the faster we can fix the issue. Check out [How to Report Bugs Effectively](http://www.chiark.greenend.org.uk/~sgtatham/bugs.html).
Please be kind, remember that Gitea comes at no cost to you, and you're getting free help.
## Discuss your design
The project welcomes submissions but please let everyone know what you're working on if you want to change or add something to the Gitea repositories.
Before starting to write something new for the Gitea project, please [file an issue](https://github.com/go-gitea/gitea/issues/new). Significant changes must go through the [change proposal process](https://github.com/go-gitea/proposals) before they can be accepted.
This process gives everyone a chance to validate the design, helps prevent duplication of effort, and ensures that the idea fits inside the goals for the project and tools. It also checks that the design is sound before code is written; the code review tool is not the place for high-level discussions.
## Testing redux
Before sending code out for review, run all the tests for the whole tree to make sure the changes don't break other usage and keep the compatibility on upgrade. To make sure you are running the test suite exactly like we do, you should install the CLI for [Drone CI](https://github.com/drone/drone), as we are using the server for continous testing, following [these instructions](http://readme.drone.io/0.5/install/cli/). After that you can simply call `drone exec` within your working directory and it will try to run the test suite locally.
## Code review
Changes to Gitea must be reviewed before they are accepted, no matter who makes the change even if it is an owner or a maintainer. We use GitHub's pull request workflow to do that and we also use [LGTM](http://lgtm.co) to ensure every PR is reviewed by at least 2 maintainers.
Please try to make your pull request easy to review for us. Please read the "[How to get faster PR reviews](https://github.com/kubernetes/community/blob/master/contributors/devel/faster_reviews.md)" guide, it has lots of useful tips for any project you may want to contribute. Some of the key points:
* Make small pull requests. The smaller, the faster to review and the more likely it will be merged soon.
* Don't make changes unrelated to your PR. Maybe there are typos on some comments, maybe refactoring would be welcome on a function... but if that is not related to your PR, please make *another* PR for that.
* Split big pull requests into multiple small ones. An incremental change will be faster to review than a huge PR.
## Sign your work
The sign-off is a simple line at the end of the explanation for the patch. Your signature certifies that you wrote the patch or otherwise have the right to pass it on as an open-source patch. The rules are pretty simple: If you can certify [DCO](DCO), then you just add a line to every git commit message:
```
Signed-off-by: Joe Smith <joe.smith@email.com>
```
Please use your real name, we really dislike pseudonyms or anonymous contributions. We are in the open-source world without secrets. If you set your `user.name` and `user.email` git configs, you can sign your commit automatically with `git commit -s`.
## Maintainers
To make sure every PR is checked, we have [team maintainers](https://github.com/orgs/go-gitea/teams/maintainers). Every PR **MUST** be reviewed by at least two maintainers (or owners) before it can get merged. A maintainer should be a contributor of Gitea (or Gogs) and contributed at least 4 accepted PRs. A contributor should apply as a maintainer in the [Discord #develop channel](https://discord.gg/NsatcWJ). The owners or the team maintainers may invite the contributor. A maintainer should spend some time on code reviews. If a maintainer has no time to do that, they should apply to leave the maintainers team and we will give them the honor of being a member of the [advisors team](https://github.com/orgs/go-gitea/teams/advisors). Of course, if an advisor has time to code review, we will gladly welcome them back to the maintainers team. If a maintainer is inactive for more than 3 months and forgets to leave the maintainers team, the owners may move him or her from the maintainers team to the advisors team.
## Owners
Since Gitea is a pure community organization without any company support, to keep the development healthy we will elect three owners every year. All contributors may vote to elect up to three candidates, one of which will be the main owner, and the other two the assistant owners. When the new owners have been elected, the old owners will give up ownership to the newly elected owners. If an owner is unable to do so, the other owners will assist in ceding ownership to the newly elected owners.
After the election, the new owners should proactively agree with our [CONTRIBUTING](CONTRIBUTING.md) requirements on the [Discord #general channel](https://discord.gg/NsatcWJ). Below are the words to speak:
```
I'm honored to having been elected an owner of Gitea, I agree with [CONTRIBUTING](CONTRIBUTING.md). I will spend part of my time on Gitea and lead the development of Gitea.
```
To honor the past owners, here's the history of the owners and the time they served:
* 2016-11-04 ~ 2017-12-31
* [Lunny Xiao](https://github.com/lunny) <xiaolunwen@gmail.com>
* [Thomas Boerger](https://github.com/tboerger) <thomas@webhippie.de>
* [Kim Carlbäcker](https://github.com/bkcsoft) <kim.carlbacker@gmail.com>
## Versions
Gitea has the `master` branch as a tip branch and has version branches such as `v0.9`. `v0.9` is a release branch and we will tag `v0.9.0` for binary download. If `v0.9.0` has bugs, we will accept pull requests on the `v0.9` branch and publish a `v0.9.1` tag, after bringing the bug fix also to the master branch.
Since the `master` branch is a tip version, if you wish to use Gitea in production, please download the latest release tag version. All the branches will be protected via GitHub, all the PRs to every branch must be reviewed by two maintainers and must pass the automatic tests.
## Copyright
Code that you contribute should use the standard copyright header:
```
// Copyright 2017 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
```
Files in the repository contain copyright from the year they are added to the year they are last changed. If the copyright author is changed, just paste the header below the old one.

36
vendor/code.gitea.io/git/DCO generated vendored
View File

@ -1,36 +0,0 @@
Developer Certificate of Origin
Version 1.1
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
660 York Street, Suite 102,
San Francisco, CA 94110 USA
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.

21
vendor/code.gitea.io/git/MAINTAINERS generated vendored
View File

@ -1,21 +0,0 @@
Alexey Makhov <amakhov@avito.ru> (@makhov)
Andrey Nering <andrey.nering@gmail.com> (@andreynering)
Bo-Yi Wu <appleboy.tw@gmail.com> (@appleboy)
Ethan Koenig <ethantkoenig@gmail.com> (@ethantkoenig)
Kees de Vries <bouwko@gmail.com> (@Bwko)
Kim Carlbäcker <kim.carlbacker@gmail.com> (@bkcsoft)
LefsFlare <nobody@nobody.tld> (@LefsFlarey)
Lunny Xiao <xiaolunwen@gmail.com> (@lunny)
Matthias Loibl <mail@matthiasloibl.com> (@metalmatze)
Morgan Bazalgette <the@howl.moe> (@thehowl)
Rachid Zarouali <nobody@nobody.tld> (@xinity)
Rémy Boulanouar <admin@dblk.org> (@DblK)
Sandro Santilli <strk@kbt.io> (@strk)
Thibault Meyer <meyer.thibault@gmail.com> (@0xbaadf00d)
Thomas Boerger <thomas@webhippie.de> (@tboerger)
Lauris Bukšis-Haberkorns <lauris@nix.lv> (@lafriks)
Antoine Girard <sapk@sapk.fr> (@sapk)
Jonas Östanbäck <jonas.ostanback@gmail.com> (@cez81)
David Schneiderbauer <dschneiderbauer@gmail.com> (@daviian)
Peter Žeby <morlinest@gmail.com> (@morlinest)
Jonas Franz <info@jonasfranz.software> (@JonasFranzDEV)

44
vendor/code.gitea.io/git/Makefile generated vendored
View File

@ -1,44 +0,0 @@
IMPORT := code.gitea.io/git
PACKAGES ?= $(shell go list -e ./... | grep -v /vendor/ | grep -v /benchmark/)
GENERATE ?= code.gitea.io/git
.PHONY: all
all: clean test build
.PHONY: clean
clean:
go clean -i ./...
generate:
@which mockery > /dev/null; if [ $$? -ne 0 ]; then \
go get -u github.com/vektra/mockery/...; \
fi
go generate $(GENERATE)
.PHONY: fmt
fmt:
find . -name "*.go" -type f ! -path "./vendor/*" ! -path "./benchmark/*" | xargs gofmt -s -w
.PHONY: vet
vet:
go vet $(PACKAGES)
.PHONY: lint
lint:
@which golint > /dev/null; if [ $$? -ne 0 ]; then \
go get -u github.com/golang/lint/golint; \
fi
for PKG in $(PACKAGES); do golint -set_exit_status $$PKG || exit 1; done;
.PHONY: test
test:
for PKG in $(PACKAGES); do go test -cover -coverprofile $$GOPATH/src/$$PKG/coverage.out $$PKG || exit 1; done;
.PHONY: bench
bench:
go test -run=XXXXXX -benchtime=10s -bench=. || exit 1
.PHONY: build
build:
go build .

25
vendor/code.gitea.io/git/README.md generated vendored
View File

@ -1,25 +0,0 @@
# Git Shell
[![Build Status](http://drone.gitea.io/api/badges/go-gitea/git/status.svg)](http://drone.gitea.io/go-gitea/git)
[![Join the chat at https://img.shields.io/discord/322538954119184384.svg](https://img.shields.io/discord/322538954119184384.svg)](https://discord.gg/NsatcWJ)
[![](https://images.microbadger.com/badges/image/gitea/gitea.svg)](http://microbadger.com/images/gitea/gitea "Get your own image badge on microbadger.com")
[![Coverage Status](https://coverage.gitea.io/badges/go-gitea/git/coverage.svg)](https://coverage.gitea.io/go-gitea/git)
[![Go Report Card](https://goreportcard.com/badge/code.gitea.io/git)](https://goreportcard.com/report/code.gitea.io/git)
[![GoDoc](https://godoc.org/code.gitea.io/git?status.svg)](https://godoc.org/code.gitea.io/git)
This project is a Go module to access Git through shell commands. For further
informations take a look at the current [documentation](https://godoc.org/code.gitea.io/git).
## Contributing
Fork -> Patch -> Push -> Pull Request
## Authors
* [Maintainers](https://github.com/orgs/go-gitea/people)
* [Contributors](https://github.com/go-gitea/git/graphs/contributors)
## License
This project is under the MIT License. See the [LICENSE](LICENSE) file for the
full license text.

View File

@ -4,7 +4,21 @@
package git
import "fmt"
// FileBlame return the Blame object of file
func (repo *Repository) FileBlame(revision, path, file string) ([]byte, error) {
return NewCommand("blame", "--root", file).RunInDirBytes(path)
return NewCommand("blame", "--root", "--", file).RunInDirBytes(path)
}
// LineBlame returns the latest commit at the given line
func (repo *Repository) LineBlame(revision, path, file string, line uint) (*Commit, error) {
res, err := NewCommand("blame", fmt.Sprintf("-L %d,%d", line, line), "-p", revision, "--", file).RunInDir(path)
if err != nil {
return nil, err
}
if len(res) < 40 {
return nil, fmt.Errorf("invalid result of blame: %s", res)
}
return repo.GetCommit(string(res[:40]))
}

View File

@ -1,126 +0,0 @@
# goquery - a little like that j-thing, only in Go
[![build status](https://secure.travis-ci.org/PuerkitoBio/goquery.png)](http://travis-ci.org/PuerkitoBio/goquery) [![GoDoc](https://godoc.org/github.com/PuerkitoBio/goquery?status.png)](http://godoc.org/github.com/PuerkitoBio/goquery) [![Sourcegraph Badge](https://sourcegraph.com/github.com/PuerkitoBio/goquery/-/badge.svg)](https://sourcegraph.com/github.com/PuerkitoBio/goquery?badge)
goquery brings a syntax and a set of features similar to [jQuery][] to the [Go language][go]. It is based on Go's [net/html package][html] and the CSS Selector library [cascadia][]. Since the net/html parser returns nodes, and not a full-featured DOM tree, jQuery's stateful manipulation functions (like height(), css(), detach()) have been left off.
Also, because the net/html parser requires UTF-8 encoding, so does goquery: it is the caller's responsibility to ensure that the source document provides UTF-8 encoded HTML. See the [wiki][] for various options to do this.
Syntax-wise, it is as close as possible to jQuery, with the same function names when possible, and that warm and fuzzy chainable interface. jQuery being the ultra-popular library that it is, I felt that writing a similar HTML-manipulating library was better to follow its API than to start anew (in the same spirit as Go's `fmt` package), even though some of its methods are less than intuitive (looking at you, [index()][index]...).
## Installation
Please note that because of the net/html dependency, goquery requires Go1.1+.
$ go get github.com/PuerkitoBio/goquery
(optional) To run unit tests:
$ cd $GOPATH/src/github.com/PuerkitoBio/goquery
$ go test
(optional) To run benchmarks (warning: it runs for a few minutes):
$ cd $GOPATH/src/github.com/PuerkitoBio/goquery
$ go test -bench=".*"
## Changelog
**Note that goquery's API is now stable, and will not break.**
* **2017-02-12 (v1.1.0)** : Add `SetHtml` and `SetText` (thanks to @glebtv).
* **2016-12-29 (v1.0.2)** : Optimize allocations for `Selection.Text` (thanks to @radovskyb).
* **2016-08-28 (v1.0.1)** : Optimize performance for large documents.
* **2016-07-27 (v1.0.0)** : Tag version 1.0.0.
* **2016-06-15** : Invalid selector strings internally compile to a `Matcher` implementation that never matches any node (instead of a panic). So for example, `doc.Find("~")` returns an empty `*Selection` object.
* **2016-02-02** : Add `NodeName` utility function similar to the DOM's `nodeName` property. It returns the tag name of the first element in a selection, and other relevant values of non-element nodes (see godoc for details). Add `OuterHtml` utility function similar to the DOM's `outerHTML` property (named `OuterHtml` in small caps for consistency with the existing `Html` method on the `Selection`).
* **2015-04-20** : Add `AttrOr` helper method to return the attribute's value or a default value if absent. Thanks to [piotrkowalczuk][piotr].
* **2015-02-04** : Add more manipulation functions - Prepend* - thanks again to [Andrew Stone][thatguystone].
* **2014-11-28** : Add more manipulation functions - ReplaceWith*, Wrap* and Unwrap - thanks again to [Andrew Stone][thatguystone].
* **2014-11-07** : Add manipulation functions (thanks to [Andrew Stone][thatguystone]) and `*Matcher` functions, that receive compiled cascadia selectors instead of selector strings, thus avoiding potential panics thrown by goquery via `cascadia.MustCompile` calls. This results in better performance (selectors can be compiled once and reused) and more idiomatic error handling (you can handle cascadia's compilation errors, instead of recovering from panics, which had been bugging me for a long time). Note that the actual type expected is a `Matcher` interface, that `cascadia.Selector` implements. Other matcher implementations could be used.
* **2014-11-06** : Change import paths of net/html to golang.org/x/net/html (see https://groups.google.com/forum/#!topic/golang-nuts/eD8dh3T9yyA). Make sure to update your code to use the new import path too when you call goquery with `html.Node`s.
* **v0.3.2** : Add `NewDocumentFromReader()` (thanks jweir) which allows creating a goquery document from an io.Reader.
* **v0.3.1** : Add `NewDocumentFromResponse()` (thanks assassingj) which allows creating a goquery document from an http response.
* **v0.3.0** : Add `EachWithBreak()` which allows to break out of an `Each()` loop by returning false. This function was added instead of changing the existing `Each()` to avoid breaking compatibility.
* **v0.2.1** : Make go-getable, now that [go.net/html is Go1.0-compatible][gonet] (thanks to @matrixik for pointing this out).
* **v0.2.0** : Add support for negative indices in Slice(). **BREAKING CHANGE** `Document.Root` is removed, `Document` is now a `Selection` itself (a selection of one, the root element, just like `Document.Root` was before). Add jQuery's Closest() method.
* **v0.1.1** : Add benchmarks to use as baseline for refactorings, refactor Next...() and Prev...() methods to use the new html package's linked list features (Next/PrevSibling, FirstChild). Good performance boost (40+% in some cases).
* **v0.1.0** : Initial release.
## API
goquery exposes two structs, `Document` and `Selection`, and the `Matcher` interface. Unlike jQuery, which is loaded as part of a DOM document, and thus acts on its containing document, goquery doesn't know which HTML document to act upon. So it needs to be told, and that's what the `Document` type is for. It holds the root document node as the initial Selection value to manipulate.
jQuery often has many variants for the same function (no argument, a selector string argument, a jQuery object argument, a DOM element argument, ...). Instead of exposing the same features in goquery as a single method with variadic empty interface arguments, statically-typed signatures are used following this naming convention:
* When the jQuery equivalent can be called with no argument, it has the same name as jQuery for the no argument signature (e.g.: `Prev()`), and the version with a selector string argument is called `XxxFiltered()` (e.g.: `PrevFiltered()`)
* When the jQuery equivalent **requires** one argument, the same name as jQuery is used for the selector string version (e.g.: `Is()`)
* The signatures accepting a jQuery object as argument are defined in goquery as `XxxSelection()` and take a `*Selection` object as argument (e.g.: `FilterSelection()`)
* The signatures accepting a DOM element as argument in jQuery are defined in goquery as `XxxNodes()` and take a variadic argument of type `*html.Node` (e.g.: `FilterNodes()`)
* The signatures accepting a function as argument in jQuery are defined in goquery as `XxxFunction()` and take a function as argument (e.g.: `FilterFunction()`)
* The goquery methods that can be called with a selector string have a corresponding version that take a `Matcher` interface and are defined as `XxxMatcher()` (e.g.: `IsMatcher()`)
Utility functions that are not in jQuery but are useful in Go are implemented as functions (that take a `*Selection` as parameter), to avoid a potential naming clash on the `*Selection`'s methods (reserved for jQuery-equivalent behaviour).
The complete [godoc reference documentation can be found here][doc].
Please note that Cascadia's selectors do not necessarily match all supported selectors of jQuery (Sizzle). See the [cascadia project][cascadia] for details. Invalid selector strings compile to a `Matcher` that fails to match any node. Behaviour of the various functions that take a selector string as argument follows from that fact, e.g. (where `~` is an invalid selector string):
* `Find("~")` returns an empty selection because the selector string doesn't match anything.
* `Add("~")` returns a new selection that holds the same nodes as the original selection, because it didn't add any node (selector string didn't match anything).
* `ParentsFiltered("~")` returns an empty selection because the selector string doesn't match anything.
* `ParentsUntil("~")` returns all parents of the selection because the selector string didn't match any element to stop before the top element.
## Examples
See some tips and tricks in the [wiki][].
Adapted from example_test.go:
```Go
package main
import (
"fmt"
"log"
"github.com/PuerkitoBio/goquery"
)
func ExampleScrape() {
doc, err := goquery.NewDocument("http://metalsucks.net")
if err != nil {
log.Fatal(err)
}
// Find the review items
doc.Find(".sidebar-reviews article .content-block").Each(func(i int, s *goquery.Selection) {
// For each item found, get the band and title
band := s.Find("a").Text()
title := s.Find("i").Text()
fmt.Printf("Review %d: %s - %s\n", i, band, title)
})
}
func main() {
ExampleScrape()
}
```
## License
The [BSD 3-Clause license][bsd], the same as the [Go language][golic]. Cascadia's license is [here][caslic].
[jquery]: http://jquery.com/
[go]: http://golang.org/
[cascadia]: https://github.com/andybalholm/cascadia
[bsd]: http://opensource.org/licenses/BSD-3-Clause
[golic]: http://golang.org/LICENSE
[caslic]: https://github.com/andybalholm/cascadia/blob/master/LICENSE
[doc]: http://godoc.org/github.com/PuerkitoBio/goquery
[index]: http://api.jquery.com/index/
[gonet]: https://github.com/golang/net/
[html]: http://godoc.org/golang.org/x/net/html
[wiki]: https://github.com/PuerkitoBio/goquery/wiki/Tips-and-tricks
[thatguystone]: https://github.com/thatguystone
[piotr]: https://github.com/piotrkowalczuk

View File

@ -1,121 +0,0 @@
.PHONY: help all test format fmtcheck vet lint qa deps clean nuke rle backrle ser fetch-real-roaring-datasets
# Display general help about this command
help:
@echo ""
@echo "The following commands are available:"
@echo ""
@echo " make qa : Run all the tests"
@echo " make test : Run the unit tests"
@echo ""
@echo " make format : Format the source code"
@echo " make fmtcheck : Check if the source code has been formatted"
@echo " make vet : Check for suspicious constructs"
@echo " make lint : Check for style errors"
@echo ""
@echo " make deps : Get the dependencies"
@echo " make clean : Remove any build artifact"
@echo " make nuke : Deletes any intermediate file"
@echo ""
@echo " make fuzz-smat : Fuzzy testing with smat"
@echo " make fuzz-stream : Fuzzy testing with stream deserialization"
@echo " make fuzz-buffer : Fuzzy testing with buffer deserialization"
@echo ""
# Alias for help target
all: help
test:
go test
go test -race -run TestConcurrent*
# Format the source code
format:
@find ./ -type f -name "*.go" -exec gofmt -w {} \;
# Check if the source code has been formatted
fmtcheck:
@mkdir -p target
@find ./ -type f -name "*.go" -exec gofmt -d {} \; | tee target/format.diff
@test ! -s target/format.diff || { echo "ERROR: the source code has not been formatted - please use 'make format' or 'gofmt'"; exit 1; }
# Check for syntax errors
vet:
GOPATH=$(GOPATH) go vet ./...
# Check for style errors
lint:
GOPATH=$(GOPATH) PATH=$(GOPATH)/bin:$(PATH) golint ./...
# Alias to run all quality-assurance checks
qa: fmtcheck test vet lint
# --- INSTALL ---
# Get the dependencies
deps:
GOPATH=$(GOPATH) go get github.com/smartystreets/goconvey/convey
GOPATH=$(GOPATH) go get github.com/willf/bitset
GOPATH=$(GOPATH) go get github.com/golang/lint/golint
GOPATH=$(GOPATH) go get github.com/mschoch/smat
GOPATH=$(GOPATH) go get github.com/dvyukov/go-fuzz/go-fuzz
GOPATH=$(GOPATH) go get github.com/dvyukov/go-fuzz/go-fuzz-build
GOPATH=$(GOPATH) go get github.com/glycerine/go-unsnap-stream
GOPATH=$(GOPATH) go get github.com/philhofer/fwd
GOPATH=$(GOPATH) go get github.com/jtolds/gls
fuzz-smat:
go test -tags=gofuzz -run=TestGenerateSmatCorpus
go-fuzz-build -func FuzzSmat github.com/RoaringBitmap/roaring
go-fuzz -bin=./roaring-fuzz.zip -workdir=workdir/ -timeout=200
fuzz-stream:
go-fuzz-build -func FuzzSerializationStream github.com/RoaringBitmap/roaring
go-fuzz -bin=./roaring-fuzz.zip -workdir=workdir/ -timeout=200
fuzz-buffer:
go-fuzz-build -func FuzzSerializationBuffer github.com/RoaringBitmap/roaring
go-fuzz -bin=./roaring-fuzz.zip -workdir=workdir/ -timeout=200
# Remove any build artifact
clean:
GOPATH=$(GOPATH) go clean ./...
# Deletes any intermediate file
nuke:
rm -rf ./target
GOPATH=$(GOPATH) go clean -i ./...
rle:
cp rle.go rle16.go
perl -pi -e 's/32/16/g' rle16.go
cp rle_test.go rle16_test.go
perl -pi -e 's/32/16/g' rle16_test.go
backrle:
cp rle16.go rle.go
perl -pi -e 's/16/32/g' rle.go
perl -pi -e 's/2032/2016/g' rle.go
ser: rle
go generate
cover:
go test -coverprofile=coverage.out
go tool cover -html=coverage.out
fetch-real-roaring-datasets:
# pull github.com/RoaringBitmap/real-roaring-datasets -> testdata/real-roaring-datasets
git submodule init
git submodule update

View File

@ -1,246 +0,0 @@
roaring [![Build Status](https://travis-ci.org/RoaringBitmap/roaring.png)](https://travis-ci.org/RoaringBitmap/roaring) [![Coverage Status](https://coveralls.io/repos/github/RoaringBitmap/roaring/badge.svg?branch=master)](https://coveralls.io/github/RoaringBitmap/roaring?branch=master) [![GoDoc](https://godoc.org/github.com/RoaringBitmap/roaring?status.svg)](https://godoc.org/github.com/RoaringBitmap/roaring) [![Go Report Card](https://goreportcard.com/badge/RoaringBitmap/roaring)](https://goreportcard.com/report/github.com/RoaringBitmap/roaring)
=============
This is a go version of the Roaring bitmap data structure.
Roaring bitmaps are used by several major systems such as [Apache Lucene][lucene] and derivative systems such as [Solr][solr] and
[Elasticsearch][elasticsearch], [Metamarkets' Druid][druid], [LinkedIn Pinot][pinot], [Netflix Atlas][atlas], [Apache Spark][spark], [OpenSearchServer][opensearchserver], [Cloud Torrent][cloudtorrent], [Whoosh][whoosh], [Pilosa][pilosa], [Microsoft Visual Studio Team Services (VSTS)][vsts], and eBay's [Apache Kylin][kylin].
[lucene]: https://lucene.apache.org/
[solr]: https://lucene.apache.org/solr/
[elasticsearch]: https://www.elastic.co/products/elasticsearch
[druid]: http://druid.io/
[spark]: https://spark.apache.org/
[opensearchserver]: http://www.opensearchserver.com
[cloudtorrent]: https://github.com/jpillora/cloud-torrent
[whoosh]: https://bitbucket.org/mchaput/whoosh/wiki/Home
[pilosa]: https://www.pilosa.com/
[kylin]: http://kylin.apache.org/
[pinot]: http://github.com/linkedin/pinot/wiki
[vsts]: https://www.visualstudio.com/team-services/
[atlas]: https://github.com/Netflix/atlas
Roaring bitmaps are found to work well in many important applications:
> Use Roaring for bitmap compression whenever possible. Do not use other bitmap compression methods ([Wang et al., SIGMOD 2017](http://db.ucsd.edu/wp-content/uploads/2017/03/sidm338-wangA.pdf))
The ``roaring`` Go library is used by
* [Cloud Torrent](https://github.com/jpillora/cloud-torrent): a self-hosted remote torrent client
* [runv](https://github.com/hyperhq/runv): an Hypervisor-based runtime for the Open Containers Initiative
* [InfluxDB](https://www.influxdata.com)
* [Pilosa](https://www.pilosa.com/)
* [Bleve](http://www.blevesearch.com)
This library is used in production in several systems, it is part of the [Awesome Go collection](https://awesome-go.com).
There are also [Java](https://github.com/RoaringBitmap/RoaringBitmap) and [C/C++](https://github.com/RoaringBitmap/CRoaring) versions. The Java, C, C++ and Go version are binary compatible: e.g, you can save bitmaps
from a Java program and load them back in Go, and vice versa. We have a [format specification](https://github.com/RoaringBitmap/RoaringFormatSpec).
This code is licensed under Apache License, Version 2.0 (ASL2.0).
Copyright 2016-... by the authors.
### References
- Daniel Lemire, Owen Kaser, Nathan Kurz, Luca Deri, Chris O'Hara, François Saint-Jacques, Gregory Ssi-Yan-Kai, Roaring Bitmaps: Implementation of an Optimized Software Library, Software: Practice and Experience 48 (4), 2018 [arXiv:1709.07821](https://arxiv.org/abs/1709.07821)
- Samy Chambi, Daniel Lemire, Owen Kaser, Robert Godin,
Better bitmap performance with Roaring bitmaps,
Software: Practice and Experience 46 (5), 2016.
http://arxiv.org/abs/1402.6407 This paper used data from http://lemire.me/data/realroaring2014.html
- Daniel Lemire, Gregory Ssi-Yan-Kai, Owen Kaser, Consistently faster and smaller compressed bitmaps with Roaring, Software: Practice and Experience 46 (11), 2016. http://arxiv.org/abs/1603.06549
### Dependencies
Dependencies are fetched automatically by giving the `-t` flag to `go get`.
they include
- github.com/smartystreets/goconvey/convey
- github.com/willf/bitset
- github.com/mschoch/smat
- github.com/glycerine/go-unsnap-stream
- github.com/philhofer/fwd
- github.com/jtolds/gls
Note that the smat library requires Go 1.6 or better.
#### Installation
- go get -t github.com/RoaringBitmap/roaring
### Example
Here is a simplified but complete example:
```go
package main
import (
"fmt"
"github.com/RoaringBitmap/roaring"
"bytes"
)
func main() {
// example inspired by https://github.com/fzandona/goroar
fmt.Println("==roaring==")
rb1 := roaring.BitmapOf(1, 2, 3, 4, 5, 100, 1000)
fmt.Println(rb1.String())
rb2 := roaring.BitmapOf(3, 4, 1000)
fmt.Println(rb2.String())
rb3 := roaring.New()
fmt.Println(rb3.String())
fmt.Println("Cardinality: ", rb1.GetCardinality())
fmt.Println("Contains 3? ", rb1.Contains(3))
rb1.And(rb2)
rb3.Add(1)
rb3.Add(5)
rb3.Or(rb1)
// computes union of the three bitmaps in parallel using 4 workers
roaring.ParOr(4, rb1, rb2, rb3)
// computes intersection of the three bitmaps in parallel using 4 workers
roaring.ParAnd(4, rb1, rb2, rb3)
// prints 1, 3, 4, 5, 1000
i := rb3.Iterator()
for i.HasNext() {
fmt.Println(i.Next())
}
fmt.Println()
// next we include an example of serialization
buf := new(bytes.Buffer)
rb1.WriteTo(buf) // we omit error handling
newrb:= roaring.New()
newrb.ReadFrom(buf)
if rb1.Equals(newrb) {
fmt.Println("I wrote the content to a byte stream and read it back.")
}
}
```
If you wish to use serialization and handle errors, you might want to
consider the following sample of code:
```go
rb := BitmapOf(1, 2, 3, 4, 5, 100, 1000)
buf := new(bytes.Buffer)
size,err:=rb.WriteTo(buf)
if err != nil {
t.Errorf("Failed writing")
}
newrb:= New()
size,err=newrb.ReadFrom(buf)
if err != nil {
t.Errorf("Failed reading")
}
if ! rb.Equals(newrb) {
t.Errorf("Cannot retrieve serialized version")
}
```
Given N integers in [0,x), then the serialized size in bytes of
a Roaring bitmap should never exceed this bound:
`` 8 + 9 * ((long)x+65535)/65536 + 2 * N ``
That is, given a fixed overhead for the universe size (x), Roaring
bitmaps never use more than 2 bytes per integer. You can call
``BoundSerializedSizeInBytes`` for a more precise estimate.
### Documentation
Current documentation is available at http://godoc.org/github.com/RoaringBitmap/roaring
### Goroutine safety
In general, it should not generally be considered safe to access
the same bitmaps using different goroutines--they are left
unsynchronized for performance. Should you want to access
a Bitmap from more than one goroutine, you should
provide synchronization. Typically this is done by using channels to pass
the *Bitmap around (in Go style; so there is only ever one owner),
or by using `sync.Mutex` to serialize operations on Bitmaps.
### Coverage
We test our software. For a report on our test coverage, see
https://coveralls.io/github/RoaringBitmap/roaring?branch=master
### Benchmark
Type
go test -bench Benchmark -run -
To run benchmarks on [Real Roaring Datasets](https://github.com/RoaringBitmap/real-roaring-datasets)
run the following:
```sh
go get github.com/RoaringBitmap/real-roaring-datasets
BENCH_REAL_DATA=1 go test -bench BenchmarkRealData -run -
```
### Iterative use
You can use roaring with gore:
- go get -u github.com/motemen/gore
- Make sure that ``$GOPATH/bin`` is in your ``$PATH``.
- go get github/RoaringBitmap/roaring
```go
$ gore
gore version 0.2.6 :help for help
gore> :import github.com/RoaringBitmap/roaring
gore> x:=roaring.New()
gore> x.Add(1)
gore> x.String()
"{1}"
```
### Fuzzy testing
You can help us test further the library with fuzzy testing:
go get github.com/dvyukov/go-fuzz/go-fuzz
go get github.com/dvyukov/go-fuzz/go-fuzz-build
go test -tags=gofuzz -run=TestGenerateSmatCorpus
go-fuzz-build github.com/RoaringBitmap/roaring
go-fuzz -bin=./roaring-fuzz.zip -workdir=workdir/ -timeout=200
Let it run, and if the # of crashers is > 0, check out the reports in
the workdir where you should be able to find the panic goroutine stack
traces.
### Alternative in Go
There is a Go version wrapping the C/C++ implementation https://github.com/RoaringBitmap/gocroaring
For an alternative implementation in Go, see https://github.com/fzandona/goroar
The two versions were written independently.
### Mailing list/discussion group
https://groups.google.com/forum/#!forum/roaring-bitmaps

View File

@ -1,67 +0,0 @@
# Govarint
This project aims to provide a simple API for the performant encoding and decoding of 32 and 64 bit integers using a variety of algorithms.
[![](http://i.imgur.com/mpgC23U.jpg)](https://www.flickr.com/photos/tsevis/8648521649/)
## Usage
Each integer encoding algorithm conforms to an encoding and decoding interface.
The interfaces also specify the size of the unsigned integer, either 32 or 64 bits, and will be referred to as XX below.
To create an encoder:
NewU32Base128Encoder(w io.Writer)
NewU64Base128Encoder(w io.Writer)
NewU32GroupVarintEncoder(w io.Writer)
For encoders, the only two commands are `PutUXX` and `Close`.
`Close` must be called as some integer encoding algorithms write in multiples.
var buf bytes.Buffer
enc := NewU32Base128Encoder(&buf)
enc.PutU32(117)
enc.PutU32(343)
enc.Close()
To create a decoder:
NewU32Base128Decoder(r io.ByteReader)
NewU64Base128Decoder(r io.ByteReader)
NewU32GroupVarintDecoder(r io.ByteReader)
For decoders, the only command is `GetUXX`.
`GetUXX` returns the value and any potential errors.
When reading is complete, `GetUXX` will return an `EOF` (End Of File).
dec := NewU32Base128Decoder(&buf)
x, err := dec.GetU32()
## Use Cases
Using fixed width integers, such as uint32 and uint64, usually waste large amounts of space, especially when encoding small values.
Optimally, smaller numbers should take less space to represent.
Using integer encoding algorithms is especially common in specific applications, such as storing edge lists or indexes for search engines.
In these situations, you have a sorted list of numbers that you want to keep as compactly as possible in memory.
Additionally, by storing only the difference between the given number and the previous (delta encoding), the numbers are quite small, and thus compress well.
For an explicit example, the Web Data Commons Hyperlink Graph contains 128 billion edges linking page A to page B, where each page is represented by a 32 bit integer.
By converting all these edges to 64 bit integers (32 | 32), sorting them, and then using delta encoding, memory usage can be reduced from 64 bits per edge down to only 9 bits per edge using the Base128 integer encoding algorithm.
This figure improves even further if compressed using conventional compression algorithms (3 bits per edge).
## Encodings supported
`govarint` supports:
+ Base128 [32, 64] - each byte uses 7 bits for encoding the integer and 1 bit for indicating if the integer requires another byte
+ Group Varint [32] - integers are encoded in blocks of four - one byte encodes the size of the following four integers, then the values of the four integers follows
Group Varint consistently beats Base128 in decompression speed but Base128 may offer improved compression ratios depending on the distribution of the supplied integers.
## Tests
go test -v -bench=.
## License
MIT License, as per `LICENSE`

View File

@ -1,37 +0,0 @@
Compression and Archive Extensions
==================================
[![Go Walker](http://gowalker.org/api/v1/badge)](http://gowalker.org/github.com/Unknwon/cae)
[中文文档](README_ZH.md)
Package cae implements PHP-like Compression and Archive Extensions.
But this package has some modifications depends on Go-style.
Reference: [PHP:Compression and Archive Extensions](http://www.php.net/manual/en/refs.compression.php).
Code Convention: based on [Go Code Convention](https://github.com/Unknwon/go-code-convention).
### Implementations
Package `zip`([Go Walker](http://gowalker.org/github.com/Unknwon/cae/zip)) and `tz`([Go Walker](http://gowalker.org/github.com/Unknwon/cae/tz)) both enable you to transparently read or write ZIP/TAR.GZ compressed archives and the files inside them.
- Features:
- Add file or directory from everywhere to archive, no one-to-one limitation.
- Extract part of entries, not all at once.
- Stream data directly into `io.Writer` without any file system storage.
### Test cases and Coverage
All subpackages use [GoConvey](http://goconvey.co/) to write test cases, and coverage is more than 80 percent.
### Use cases
- [Gogs](https://github.com/gogits/gogs): self hosted Git service in the Go Programming Language.
- [GoBlog](https://github.com/fuxiaohei/GoBlog): personal blogging application.
- [GoBuild](https://github.com/shxsun/gobuild/): online Go cross-platform compilation and download service.
## License
This project is under Apache v2 License. See the [LICENSE](LICENSE) file for the full license text.

View File

@ -1,29 +0,0 @@
压缩与打包扩展
=============
[![Go Walker](http://gowalker.org/api/v1/badge)](http://gowalker.org/github.com/Unknwon/cae)
包 cae 实现了 PHP 风格的压缩与打包扩展。
但本包依据 Go 语言的风格进行了一些修改。
引用:[PHP:Compression and Archive Extensions](http://www.php.net/manual/en/refs.compression.php)
编码规范:基于 [Go 编码规范](https://github.com/Unknwon/go-code-convention)
### 实现
`zip`([Go Walker](http://gowalker.org/github.com/Unknwon/cae/zip)) 和 `tz`([Go Walker](http://gowalker.org/github.com/Unknwon/cae/tz)) 都允许你轻易的读取或写入 ZIP/TAR.GZ 压缩档案和其内部文件。
- 特性:
- 将任意位置的文件或目录加入档案,没有一对一的操作限制。
- 只解压部分文件,而非一次性解压全部。
- 将数据以流的形式直接写入 `io.Writer` 而不需经过文件系统的存储。
### 测试用例与覆盖率
所有子包均采用 [GoConvey](http://goconvey.co/) 来书写测试用例,覆盖率均超过 80%。
## 授权许可
本项目采用 Apache v2 开源授权许可证,完整的授权说明已放置在 [LICENSE](LICENSE) 文件中。

1
vendor/github.com/Unknwon/cae/tz/testdata/test.lnk generated vendored Symbolic link
View File

@ -0,0 +1 @@
test.zip

1
vendor/github.com/Unknwon/cae/tz/testdata/testdir.lnk generated vendored Symbolic link
View File

@ -0,0 +1 @@
testdir

1
vendor/github.com/Unknwon/cae/zip/testdata/test.lnk generated vendored Symbolic link
View File

@ -0,0 +1 @@
test.zip

1
vendor/github.com/Unknwon/cae/zip/testdata/testdir.lnk generated vendored Symbolic link
View File

@ -0,0 +1 @@
testdir

View File

@ -1,20 +0,0 @@
Common Functions
================
[![Build Status](https://travis-ci.org/Unknwon/com.svg)](https://travis-ci.org/Unknwon/com) [![Go Walker](http://gowalker.org/api/v1/badge)](http://gowalker.org/github.com/Unknwon/com)
This is an open source project for commonly used functions for the Go programming language.
This package need >= **go 1.2**
Code Convention: based on [Go Code Convention](https://github.com/Unknwon/go-code-convention).
## Contribute
Your contribute is welcome, but you have to check following steps after you added some functions and commit them:
1. Make sure you wrote user-friendly comments for **all functions** .
2. Make sure you wrote test cases with any possible condition for **all functions** in file `*_test.go`.
3. Make sure you wrote benchmarks for **all functions** in file `*_test.go`.
4. Make sure you wrote useful examples for **all functions** in file `example_test.go`.
5. Make sure you ran `go test` and got **PASS** .

View File

@ -1,12 +0,0 @@
.PHONY: build test bench vet
build: vet bench
test:
go test -v -cover
bench:
go test -v -cover -test.bench=. -test.benchmem
vet:
go vet

View File

@ -1,136 +0,0 @@
i18n [![GoDoc](https://godoc.org/github.com/Unknwon/i18n?status.svg)](https://godoc.org/github.com/Unknwon/i18n) [![Sourcegraph](https://sourcegraph.com/github.com/Unknwon/i18n/-/badge.svg)](https://sourcegraph.com/github.com/Unknwon/i18n?badge)
====
Package i18n is for app Internationalization and Localization.
## Introduction
This package provides multiple-language options to improve user experience. Sites like [Go Walker](http://gowalker.org) and [gogs.io](http://gogs.io) are using this module to implement Chinese and English user interfaces.
You can use following command to install this module:
go get github.com/Unknwon/i18n
## Usage
First of all, you have to import this package:
```go
import "github.com/Unknwon/i18n"
```
The format of locale files is very like INI format configuration file, which is basically key-value pairs. But this module has some improvements. Every language corresponding to a locale file, for example, under `conf/locale` folder of [gogsweb](https://github.com/gogits/gogsweb/tree/master/conf/locale), there are two files called `locale_en-US.ini` and `locale_zh-CN.ini`.
The name and extensions of locale files can be anything, but we strongly recommend you to follow the style of gogsweb.
## Minimal example
Here are two simplest locale file examples:
File `locale_en-US.ini`:
```ini
hi = hello, %s
bye = goodbye
```
File `locale_zh-CN.ini`:
```ini
hi = 您好,%s
bye = 再见
```
### Do Translation
There are two ways to do translation depends on which way is the best fit for your application or framework.
Directly use package function to translate:
```go
i18n.Tr("en-US", "hi", "Unknwon")
i18n.Tr("en-US", "bye")
```
Or create a struct and embed it:
```go
type MyController struct{
// ...other fields
i18n.Locale
}
//...
func ... {
c := &MyController{
Locale: i18n.Locale{"en-US"},
}
_ = c.Tr("hi", "Unknwon")
_ = c.Tr("bye")
}
```
Code above will produce correspondingly:
- English `en-US``hello, Unknwon`, `goodbye`
- Chinese `zh-CN``您好Unknwon`, `再见`
## Section
For different pages, one key may map to different values. Therefore, i18n module also uses the section feature of INI format configuration to achieve section.
For example, the key name is `about`, and we want to show `About` in the home page and `About Us` in about page. Then you can do following:
Content in locale file:
```ini
about = About
[about]
about = About Us
```
Get `about` in home page:
```go
i18n.Tr("en-US", "about")
```
Get `about` in about page:
```go
i18n.Tr("en-US", "about.about")
```
### Ambiguity
Because dot `.` is sign of section in both [INI parser](https://github.com/go-ini/ini) and locale files, so when your key name contains `.` will cause ambiguity. At this point, you just need to add one more `.` in front of the key.
For example, the key name is `about.`, then we can use:
```go
i18n.Tr("en-US", ".about.")
```
to get expect result.
## Helper tool
Module i18n provides a command line helper tool beei18n for simplify steps of your development. You can install it as follows:
go get github.com/Unknwon/i18n/ui18n
### Sync locale files
Command `sync` allows you use a exist local file as the template to create or sync other locale files:
ui18n sync srouce_file.ini other1.ini other2.ini
This command can operate 1 or more files in one command.
## More information
- The first locale you load to the module is considered as **default locale**.
- When matching non-default locale and didn't find the string, i18n will have a second try on default locale.
- If i18n still cannot find string in the default locale, raw string will be returned. For instance, when the string is `hi` and it does not exist in locale file, simply return `hi` as output.

View File

@ -1,65 +0,0 @@
Paginater [![Build Status](https://drone.io/github.com/Unknwon/paginater/status.png)](https://drone.io/github.com/Unknwon/paginater/latest) [![](http://gocover.io/_badge/github.com/Unknwon/paginater)](http://gocover.io/github.com/Unknwon/paginater)
=========
Package paginater is a helper module for custom pagination calculation.
## Installation
go get github.com/Unknwon/paginater
## Getting Started
The following code shows an example of how to use paginater:
```go
package main
import "github.com/Unknwon/paginater"
func main() {
// Arguments:
// - Total number of rows
// - Number of rows in one page
// - Current page number
// - Number of page links
p := paginater.New(45, 10, 3, 3)
// Then use p as a template object named "Page" in "demo.html"
// ...
}
```
`demo.html`
```html
{{if not .Page.IsFirst}}[First](1){{end}}
{{if .Page.HasPrevious}}[Previous]({{.Page.Previous}}){{end}}
{{range .Page.Pages}}
{{if eq .Num -1}}
...
{{else}}
{{.Num}}{{if .IsCurrent}}(current){{end}}
{{end}}
{{end}}
{{if .Page.HasNext}}[Next]({{.Page.Next}}){{end}}
{{if not .Page.IsLast}}[Last]({{.Page.TotalPages}}){{end}}
```
Possible output:
```
[First](1) [Previous](2) ... 2 3(current) 4 ... [Next](4) [Last](5)
```
As you may guess, if the `Page` value is `-1`, you should print `...` in the HTML as common practice.
## Getting Help
- [API Documentation](https://gowalker.org/github.com/Unknwon/paginater)
- [File An Issue](https://github.com/Unknwon/paginater/issues/new)
## License
This project is under Apache v2 License. See the [LICENSE](LICENSE) file for the full license text.

View File

@ -1,7 +0,0 @@
# cascadia
[![](https://travis-ci.org/andybalholm/cascadia.svg)](https://travis-ci.org/andybalholm/cascadia)
The Cascadia package implements CSS selectors for use with the parse trees produced by the html package.
To test CSS selectors without writing Go code, check out [cascadia](https://github.com/suntong/cascadia) the command line tool, a thin wrapper around this package.

View File

@ -1,16 +0,0 @@
# Contributing to Bleve
We look forward to your contributions, but ask that you first review these guidelines.
### Sign the CLA
As Bleve is a Couchbase project we require contributors accept the [Couchbase Contributor License Agreement](http://review.couchbase.org/static/individual_agreement.html). To sign this agreement log into the Couchbase [code review tool](http://review.couchbase.org/). The Bleve project does not use this code review tool but it is still used to track acceptance of the contributor license agreements.
### Submitting a Pull Request
All types of contributions are welcome, but please keep the following in mind:
- If you're planning a large change, you should really discuss it in a github issue or on the google group first. This helps avoid duplicate effort and spending time on something that may not be merged.
- Existing tests should continue to pass, new tests for the contribution are nice to have.
- All code should have gone through `go fmt`
- All code should pass `go vet`

View File

@ -1,67 +0,0 @@
# ![bleve](docs/bleve.png) bleve
[![Build Status](https://travis-ci.org/blevesearch/bleve.svg?branch=master)](https://travis-ci.org/blevesearch/bleve) [![Coverage Status](https://coveralls.io/repos/github/blevesearch/bleve/badge.svg?branch=master)](https://coveralls.io/github/blevesearch/bleve?branch=master) [![GoDoc](https://godoc.org/github.com/blevesearch/bleve?status.svg)](https://godoc.org/github.com/blevesearch/bleve)
[![Join the chat at https://gitter.im/blevesearch/bleve](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/blevesearch/bleve?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
[![codebeat](https://codebeat.co/badges/38a7cbc9-9cf5-41c0-a315-0746178230f4)](https://codebeat.co/projects/github-com-blevesearch-bleve)
[![Go Report Card](https://goreportcard.com/badge/blevesearch/bleve)](https://goreportcard.com/report/blevesearch/bleve)
[![Sourcegraph](https://sourcegraph.com/github.com/blevesearch/bleve/-/badge.svg)](https://sourcegraph.com/github.com/blevesearch/bleve?badge) [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
modern text indexing in go - [blevesearch.com](http://www.blevesearch.com/)
Try out bleve live by [searching the bleve website](http://www.blevesearch.com/search/?q=bleve).
## Features
* Index any go data structure (including JSON)
* Intelligent defaults backed up by powerful configuration
* Supported field types:
* Text, Numeric, Date
* Supported query types:
* Term, Phrase, Match, Match Phrase, Prefix
* Conjunction, Disjunction, Boolean
* Numeric Range, Date Range
* Simple query [syntax](http://www.blevesearch.com/docs/Query-String-Query/) for human entry
* tf-idf Scoring
* Search result match highlighting
* Supports Aggregating Facets:
* Terms Facet
* Numeric Range Facet
* Date Range Facet
## Discussion
Discuss usage and development of bleve in the [google group](https://groups.google.com/forum/#!forum/bleve).
## Indexing
```go
message := struct{
Id string
From string
Body string
}{
Id: "example",
From: "marty.schoch@gmail.com",
Body: "bleve indexing is easy",
}
mapping := bleve.NewIndexMapping()
index, err := bleve.New("example.bleve", mapping)
if err != nil {
panic(err)
}
index.Index(message.Id, message)
```
## Querying
```go
index, _ := bleve.Open("example.bleve")
query := bleve.NewQueryStringQuery("bleve")
searchRequest := bleve.NewSearchRequest(query)
searchResult, _ := index.Search(searchRequest)
```
## License
Apache License Version 2.0

View File

@ -1,7 +0,0 @@
# full line comment
marty
steve # trailing comment
| different format of comment
dustin
siri | different style trailing comment
multiple words with different whitespace

View File

@ -1,9 +0,0 @@
# geo support in bleve
First, all of this geo code is a Go adaptation of the [Lucene 5.3.2 sandbox geo support](https://lucene.apache.org/core/5_3_2/sandbox/org/apache/lucene/util/package-summary.html).
## Notes
- All of the APIs will use float64 for lon/lat values.
- When describing a point in function arguments or return values, we always use the order lon, lat.
- High level APIs will use TopLeft and BottomRight to describe bounding boxes. This may not map cleanly to min/max lon/lat when crossing the dateline. The lower level APIs will use min/max lon/lat and require the higher-level code to split boxes accordingly.

File diff suppressed because it is too large Load Diff

View File

@ -1,167 +0,0 @@
# zap file format
The file is written in the reverse order that we typically access data. This helps us write in one pass since later sections of the file require file offsets of things we've already written.
Current usage:
- mmap the entire file
- crc-32 bytes and version are in fixed position at end of the file
- reading remainder of footer could be version specific
- remainder of footer gives us:
- 3 important offsets (docValue , fields index and stored data index)
- 2 important values (number of docs and chunk factor)
- field data is processed once and memoized onto the heap so that we never have to go back to disk for it
- access to stored data by doc number means first navigating to the stored data index, then accessing a fixed position offset into that slice, which gives us the actual address of the data. the first bytes of that section tell us the size of data so that we know where it ends.
- access to all other indexed data follows the following pattern:
- first know the field name -> convert to id
- next navigate to term dictionary for that field
- some operations stop here and do dictionary ops
- next use dictionary to navigate to posting list for a specific term
- walk posting list
- if necessary, walk posting details as we go
- if location info is desired, consult location bitmap to see if it is there
## stored fields section
- for each document
- preparation phase:
- produce a slice of metadata bytes and data bytes
- produce these slices in field id order
- field value is appended to the data slice
- metadata slice is govarint encoded with the following values for each field value
- field id (uint16)
- field type (byte)
- field value start offset in uncompressed data slice (uint64)
- field value length (uint64)
- field number of array positions (uint64)
- one additional value for each array position (uint64)
- compress the data slice using snappy
- file writing phase:
- remember the start offset for this document
- write out meta data length (varint uint64)
- write out compressed data length (varint uint64)
- write out the metadata bytes
- write out the compressed data bytes
## stored fields idx
- for each document
- write start offset (remembered from previous section) of stored data (big endian uint64)
With this index and a known document number, we have direct access to all the stored field data.
## posting details (freq/norm) section
- for each posting list
- produce a slice containing multiple consecutive chunks (each chunk is govarint stream)
- produce a slice remembering offsets of where each chunk starts
- preparation phase:
- for each hit in the posting list
- if this hit is in next chunk close out encoding of last chunk and record offset start of next
- encode term frequency (uint64)
- encode norm factor (float32)
- file writing phase:
- remember start position for this posting list details
- write out number of chunks that follow (varint uint64)
- write out length of each chunk (each a varint uint64)
- write out the byte slice containing all the chunk data
If you know the doc number you're interested in, this format lets you jump to the correct chunk (docNum/chunkFactor) directly and then seek within that chunk until you find it.
## posting details (location) section
- for each posting list
- produce a slice containing multiple consecutive chunks (each chunk is govarint stream)
- produce a slice remembering offsets of where each chunk starts
- preparation phase:
- for each hit in the posting list
- if this hit is in next chunk close out encoding of last chunk and record offset start of next
- encode field (uint16)
- encode field pos (uint64)
- encode field start (uint64)
- encode field end (uint64)
- encode number of array positions to follow (uint64)
- encode each array position (each uint64)
- file writing phase:
- remember start position for this posting list details
- write out number of chunks that follow (varint uint64)
- write out length of each chunk (each a varint uint64)
- write out the byte slice containing all the chunk data
If you know the doc number you're interested in, this format lets you jump to the correct chunk (docNum/chunkFactor) directly and then seek within that chunk until you find it.
## bitmaps of hits with location info
- for each posting list
- preparation phase:
- encode roaring bitmap (inidicating which hits have location details indexed) posting list to bytes (so we know the length)
- file writing phase:
- remember the start position for this bitmap
- write length of encoded roaring bitmap
- write the serialized roaring bitmap data
## postings list section
- for each posting list
- preparation phase:
- encode roaring bitmap posting list to bytes (so we know the length)
- file writing phase:
- remember the start position for this posting list
- write freq/norm details offset (remembered from previous, as varint uint64)
- write location details offset (remembered from previous, as varint uint64)
- write location bitmap offset (remembered from pervious, as varint uint64)
- write length of encoded roaring bitmap
- write the serialized roaring bitmap data
## dictionary
- for each field
- preparation phase:
- encode vellum FST with dictionary data pointing to file offset of posting list (remembered from previous)
- file writing phase:
- remember the start position of this persistDictionary
- write length of vellum data (varint uint64)
- write out vellum data
## fields section
- for each field
- file writing phase:
- remember start offset for each field
- write dictionary address (remembered from previous) (varint uint64)
- write length of field name (varint uint64)
- write field name bytes
## fields idx
- for each field
- file writing phase:
- write big endian uint64 of start offset for each field
NOTE: currently we don't know or record the length of this fields index. Instead we rely on the fact that we know it immediately precedes a footer of known size.
## fields DocValue
- for each field
- preparation phase:
- produce a slice containing multiple consecutive chunks, where each chunk is composed of a meta section followed by compressed columnar field data
- produce a slice remembering the length of each chunk
- file writing phase:
- remember the start position of this first field DocValue offset in the footer
- write out number of chunks that follow (varint uint64)
- write out length of each chunk (each a varint uint64)
- write out the byte slice containing all the chunk data
NOTE: currently the meta header inside each chunk gives clue to the location offsets and size of the data pertaining to a given docID and any
read operation leverage that meta information to extract the document specific data from the file.
## footer
- file writing phase
- write number of docs (big endian uint64)
- write stored field index location (big endian uint64)
- write field index location (big endian uint64)
- write field docValue location (big endian uint64)
- write out chunk factor (big endian uint32)
- write out version (big endian uint32)
- write out file CRC of everything preceding this (big endian uint32)

Some files were not shown because too many files have changed in this diff Show More