Ma/origin
1
0
Fork 0
mirror of https://github.com/duanhf2012/origin.git synced 2026-02-03 22:45:13 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

提交origin2.0版本

Browse Source
This commit is contained in:
duanhf2012
2020-03-28 09:57:16 +08:00
parent 0d98f77d07
commit 84fb8ab36d
111 changed files with 3657 additions and 8382 deletions
Download Patch File Download Diff File Expand all files Collapse all files

674
LICENSE
View File

@@ -1,674 +0,0 @@
GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU General Public License is a free, copyleft license for
software and other kinds of works.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
the GNU General Public License is intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users. We, the Free Software Foundation, use the
GNU General Public License for most of our software; it applies also to
any other work released this way by its authors. You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
To protect your rights, we need to prevent others from denying you
these rights or asking you to surrender the rights. Therefore, you have
certain responsibilities if you distribute copies of the software, or if
you modify it: responsibilities to respect the freedom of others.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must pass on to the recipients the same
freedoms that you received. You must make sure that they, too, receive
or can get the source code. And you must show them these terms so they
know their rights.
Developers that use the GNU GPL protect your rights with two steps:
(1) assert copyright on the software, and (2) offer you this License
giving you legal permission to copy, distribute and/or modify it.
For the developers' and authors' protection, the GPL clearly explains
that there is no warranty for this free software. For both users' and
authors' sake, the GPL requires that modified versions be marked as
changed, so that their problems will not be attributed erroneously to
authors of previous versions.
Some devices are designed to deny users access to install or run
modified versions of the software inside them, although the manufacturer
can do so. This is fundamentally incompatible with the aim of
protecting users' freedom to change the software. The systematic
pattern of such abuse occurs in the area of products for individuals to
use, which is precisely where it is most unacceptable. Therefore, we
have designed this version of the GPL to prohibit the practice for those
products. If such problems arise substantially in other domains, we
stand ready to extend this provision to those domains in future versions
of the GPL, as needed to protect the freedom of users.
Finally, every program is threatened constantly by software patents.
States should not allow patents to restrict development and use of
software on general-purpose computers, but in those that do, we wish to
avoid the special danger that patents applied to a free program could
make it effectively proprietary. To prevent this, the GPL assures that
patents cannot be used to render the program non-free.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Use with the GNU Affero General Public License.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU Affero General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the special requirements of the GNU Affero General Public License,
section 13, concerning interaction through a network will apply to the
combination as such.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If the program does terminal interaction, make it output a short
notice like this when it starts in an interactive mode:
<program> Copyright (C) <year> <name of author>
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, your program's commands
might be different; for a GUI interface, you would use an "about box".
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU GPL, see
<https://www.gnu.org/licenses/>.
The GNU General Public License does not permit incorporating your program
into proprietary programs. If your program is a subroutine library, you
may consider it more useful to permit linking proprietary applications with
the library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License. But first, please read
<https://www.gnu.org/licenses/why-not-lgpl.html>.

585
README.md
View File

@@ -1,585 +0,0 @@
origin 游戏服务器引擎简介
==================
origin 是一个由 Go 语言（golang）编写的分布式开源游戏服务器引擎。origin适用于各类游戏服务器的开发，包括 H5（HTML5）游戏服务器。
origin 解决的问题：
* origin总体设计如go语言设计一样，总是尽可能的提供简洁和易用的模式，快速开发。
* 能够根据业务需求快速并灵活的制定服务器架构。
* 利用多核优势，将不同的service配置到不同的node，并能高效的协同工作。
* 将整个引擎抽象三大对象，node,service,module。通过统一的组合模式管理游戏中各功能模块的关系。
* 有丰富并健壮的工具库。
Hello world!
---------------
下面我们来一步步的建立origin服务器,先下载[origin引擎](https://github.com/duanhf2012/origin "origin引擎"),或者使用如下命令：
```go
go get -v -u github.com/duanhf2012/origin
```
于是下载到GOPATH环境目录中,在src中加入main.go,内容如下：
```go
package main
import (
"github.com/duanhf2012/origin/originnode"
)
func main() {
node := originnode.NewOriginNode()
if node == nil {
return
}
node.Init()
node.Start()
}
```
一个origin服务器需要创建一个node对象，然后必需有Init和Start的流程。
origin引擎三大对象关系
---------------
* Node: 可以认为每一个Node代表着一个origin进程
* Service:一个独立的服务可以认为是一个大的功能模块，他是Node的子集，创建完成并安装Node对象中。服务可以支持外部RPC和HTTP接口对外功能。
* Module: 这是origin最小对象单元，强烈建议所有的业务模块都划分成各个小的Module组合。Module可以建立树状关系。Service本身也是Module的类型。
origin集群核心配置文件config/cluser.json如下:
---------------
```
{
"SubNet": [{
"Remark": "Manual,Full,Auto",
"SubNetMode": "Full",
"SubNetName": "SubNet1",
"PublicServiceList": ["logiclog"],
"NodeList": [{
"NodeID": 1,
"NodeName": "N_Node1",
"ServiceList": [
"HttpServerService",
"SubNet1_Service",
"SubNet1_Service1"
],
"ClusterNode":["SubNet2.N_Node1","N_Node2"]
},
{
"NodeID": 2,
"NodeName": "N_Node2",
"ServiceList": [
"SubNet1_Service2"
],
"ClusterNode":[]
}
]
},
{
"Remark": "Manual,Full,Auto",
"SubNetMode": "Full",
"SubNetName": "SubNet2",
"PublicServiceList": ["logiclog"],
"NodeList": [{
"NodeID": 3,
"NodeName": "N_Node1",
"ServiceList": [
"SubNet2_Service1"
],
"ClusterNode":["URQ.N_Node1"]
},
{
"NodeID": 4,
"NodeName": "N_Node4",
"ServiceList": [
"SubNet2_Service2"
],
"ClusterNode":[]
}
]
}
]
}
```
origin集群配置以子网的模式配置，在每个子网下配置多个Node服务器,子网在应对复杂的系统时可以应用到各个子系统，方便每个子系统的隔离。
origin所有的结点与服务通过配置进行关联，配置文件分为两大配置结点：
* SubNet：配置子网，以上配置中包括子网名为SubNet1与SubNet2,每个子网包含多个Node结点。
* SubNetMode：子网模式，Manual手动模式,Full通过配置全自动连接集群模式（推荐模式）,Auto自动模式
* SubNetName：子网名称
* PublicServiceList：用于公共服务配置，所有的结点默认会加载该服务列表
* SubNetMode：子网集群模式，
* NodeList：Node所有的列表
* NodeId: Node编号，用于标识唯一的进程id
* NodeName: Node名称，用于区分Node结点功能。例如，可以是GameServer
* ServiceList:结点中允许开启的服务列表
* ClusterNode:将与列表中的Node产生集群关系。允许访问这些结点中所有的服务。允许集群其他子网结点，例如：URQ.N_Node1
origin集群核心配置文件config/nodeconfig.json如下:
---------------
```
{
"Public": {
"LogLevel": 1,
"HttpPort": 9400,
"WSPort": 9500,
"CAFile": [{
"CertFile": "",
"KeyFile": ""
},
{
"CertFile": "",
"KeyFile": ""
}
],
"Environment": "FS_Dev_LFY",
"IsListenLog": 1,
"IsSendErrorMail": 0
},
"NodeList": [{
"NodeID": 1,
"NodeAddr": "127.0.0.1:8081",
"LogLevel": 1,
"HttpPort": 7001,
"WSPort": 7000,
"CertFile": "",
"KeyFile": ""
},
{
"NodeID": 2,
"NodeAddr": "127.0.0.1:8082"
},
{
"NodeID": 3,
"NodeAddr": "127.0.0.1:8083"
},
{
"NodeID": 4,
"NodeAddr": "127.0.0.1:8084"
}
]
}
```
针对cluster.json中NodeId在该文件中配置具体环境的信息
* Public：公共服务配置
* LogLevel：日志等级 1:DEBUG 2:INFO 3:WARN 4:ERROR 5:FATAL。
* HttpPort：当Node需要提供http服务时，安装HttpServerService后将使用该监听端口。
* WSPort：当Node需要提供Websocket服务时，安装HttpServerService后将使用该监听端口。
* CAFile：证书配置文件，支持多个。
* NodeList：
* NodeID：结点ID，在cluster.json中所有的Node都需要在该文件中配置。
* NodeAddr：监听RPC地址与端口。
* 其他：该配置可以继承Public中所有的配置，可以在其中自定义LogLevel,HttpPort等。
origin第一个服务:
---------------
我们准备的NodeId为1的结点下新建两个服务，分别是CTestService1与CTestService2。
* config/cluster.json内容如下
```
{
"SubNet": [{
"Remark": "Manual,Full,Auto",
"SubNetMode": "Full",
"SubNetName": "SubNet1",
"PublicServiceList": ["logiclog"],
"NodeList": [{
"NodeID": 1,
"NodeName": "N_Node1",
"ServiceList": [
"TestService1",
"TestService2"
],
"ClusterNode":[]
},
{
"NodeID": 2,
"NodeName": "N_Node2",
"ServiceList": [
"TestService3"
],
"ClusterNode":[]
}
]
}
]
}
```
* config/nodeconfig.json内容如下
```
{
"Public": {
"LogLevel": 1,
"HttpPort": 9400,
"WSPort": 9500,
"Environment": "Test",
"IsListenLog": 1,
"IsSendErrorMail": 0
},
"NodeList": [{
"NodeID": 1,
"NodeAddr": "127.0.0.1:8081"
},
{
"NodeID": 2,
"NodeAddr": "127.0.0.1:8082"
}
]
}
```
* main.go运行代码
```go
package main
import (
"github.com/duanhf2012/origin/originnode"
)
func main() {
node := originnode.NewOriginNode()
if node == nil {
return
}
node.Init()
node.Start()
}
```
* TestService1.go运行代码
```go
package main
import (
"fmt"
"github.com/duanhf2012/origin/cluster"
"github.com/duanhf2012/origin/originnode"
"github.com/duanhf2012/origin/service"
)
type TestService1 struct {
service.BaseService
}
func init() {
originnode.InitService(&TestService1{})
}
//OnInit ...
func (ws *TestService1) OnInit() error {
return nil
}
//OnRun ...
func (ws *TestService1) OnRun() bool {
var i InputData
var j int
i.A1 = 3
i.A2 = 4
err := cluster.Call("TestService2.RPC_Add", &i, &j)
if err == nil {
fmt.Printf(" TestService2.RPC_Add is %d\n", j)
}
err = cluster.Call("TestService3.RPC_Multi", &i, &j)
if err == nil {
fmt.Printf(" TestService2.RPC_Multi is %d\n", j)
}
return false
}
```
* TestService2.go运行代码
```go
package main
import (
"github.com/duanhf2012/origin/originnode"
"github.com/duanhf2012/origin/service"
)
type InputData struct {
A1 int
A2 int
}
type TestService2 struct {
service.BaseService
}
func init() {
originnode.InitService(&TestService2{})
}
//OnInit ...
func (ws *TestService2) OnInit() error {
return nil
}
//OnRun ...
func (ws *TestService2) OnRun() bool {
return false
}
//服务要对外的接口规划如下：
//RPC_MethodName(arg *DataType1, ret *DataType2) error
//如果不符合规范，在加载服务时，该函数将不会被映射，其他服务将不允能调用。
func (slf *TestService2) RPC_Add(arg *InputData, ret *int) error {
*ret = arg.A1 + arg.A2
return nil
}
```
* TestService3.go运行代码
```go
package main
import (
"github.com/duanhf2012/origin/originnode"
"github.com/duanhf2012/origin/service"
)
type TestService3 struct {
service.BaseService
}
func init() {
originnode.InitService(&TestService3{})
}
//OnInit ...
func (ws *TestService3) OnInit() error {
return nil
}
//OnRun ...
func (ws *TestService3) OnRun() bool {
return false
}
func (slf *TestService3) RPC_Multi(arg *InputData, ret *int) error {
*ret = arg.A1 * arg.A2
return nil
}
```
通过以下命令运行：
```
OriginServer.exe NodeId=2
OriginServer.exe NodeId=1
```
其中NodeId=1的进程输出结果：
```
TestService2.RPC_Add is 7
TestService2.RPC_Multi is 12
```
通过日志可以确认，在Node启动时分别驱动Service的OnInit,OnRun,OnEndRun，上面的日志中TestService1.OnRun会被调用，
因为在OnRun的返回是false，所以OnRun只会进入一次。当返回true时，会重复循环进入OnRun。如果你不需要OnRun可以不定义OnRun函数。
示例中，我们分别调用了本进程的TestService2服务中的RPC_Add的RPC接口，以及NodeId为2进程中服务为TestService3的接口RPC_Multi。
origin服务间通信:
---------------
origin是通过rpc的方式互相调用，当前结点只能访问cluster.json中有配置ClusterNode的结点或本地结点中所有的服务接口,下面我们来用实际例子来说明，如下代码所示：
```
package main
import (
"Server/service/websockservice"
"fmt"
"time"
"github.com/duanhf2012/origin/cluster"
"github.com/duanhf2012/origin/sysservice"
"github.com/duanhf2012/origin/originnode"
"github.com/duanhf2012/origin/service"
)
type CTestService1 struct {
service.BaseService
}
//输入参数，注意变量名首字母大写
type InputData struct {
A1 int
A2 int
}
func (slf *CTestService1) OnRun() bool {
var ret int
input := InputData{100, 11}
//调用服务名.接口名称，传入传出参数必需为地址，符合RPC_Add接口规范
//以下可以采用其他，注意如果在服务名前加入下划线"_"表示访问本node中的服务
//_servicename.methodname
//servicename.methodname
err := cluster.Call("CTestService2.RPC_Add", &input, &ret)
fmt.Print(err, "\n", ret)
return false
}
type CTestService2 struct {
service.BaseService
}
//注意格式一定要RPC_开头，函数第一个参数为输入参数，第二个为输出参数，只允许指针类型
//返回值必需为error，如果不满足格式，装载服务时将被会忽略。
func (slf *CTestService2) RPC_Add(arg *InputData, ret *int) error {
*ret = arg.A1 + arg.A2
return nil
}
func main() {
node := originnode.NewOriginNode()
if node == nil {
return
}
//也可以通过以下方式安装服务CTestService1与CTestService2
node.SetupService(&CTestService1{}, &CTestService2{})
node.Init()
node.Start()
}
```
输入结果为：
```
<nil>
111
```
cluster.Call只允许调用一个结点中的服务，如果服务在多个结点中，是不允许的。注意，Call方式是阻塞模式，只有当被调服务响应时才返回，或者超过最大超时时间。如果不想阻塞，可以采用Go方式调用。例如：cluster.Go(true,"CTestService2.RPC_Send", &input)
第一个参数代码是否广播，如果调用的服务接口在多个Node中存在，都将被调用。还可以向指定的NodeId调用，例如：
```
func (slf *CCluster) CallNode(nodeid int, servicemethod string, args interface{}, reply interface{}) error
func (slf *CCluster) GoNode(nodeid int, args interface{}, servicemethod string) error
```
在实际使用时，注意抽象service，只有合理的划分service，origin是以service为最小集群单元放到不同的node中，以达到动态移动service功能到不同的node进程中。
origin中Module使用:
---------------
module在origin引擎中是最小的对象单元，service本质上也是一个复杂的module。它同样有着以下方法:
```
OnInit() error //Module初始化时调用
OnRun() bool //Module运行时调用
OnEndRun()
```
在使用规则上和service是一样的，因为本质上是一样的对象。看以下简单示例：
```
package main
import (
"fmt"
"time"
"github.com/duanhf2012/origin/originnode"
"github.com/duanhf2012/origin/service"
)
type CTestService1 struct {
service.BaseService
}
type CTestModule1 struct {
service.BaseModule
}
func (slf *CTestModule1) OnInit() error {
fmt.Printf("CTestModule1::OnInit\n")
return nil
}
func (slf *CTestModule1) OnRun() bool {
fmt.Printf("CTestModule1::OnRun\n")
time.Sleep(time.Second * 1)
return true
}
func (slf *CTestModule1) OnEndRun() {
fmt.Printf("CTestModule1::OnEndRun\n")
}
func (slf *CTestModule1) Add(a int, b int) int {
return a + b
}
func (slf *CTestService1) OnRun() bool {
testmodule := CTestModule1{}
//可以设置自定义id
//testmodule.SetModuleId(PLAYERID)
//添加module到slf对象中
moduleId := slf.AddModule(&testmodule)
//获取module对象
pModule := slf.GetModuleById(moduleId)
//转换为CTestModule1类型
ret := pModule.(*CTestModule1).Add(3, 4)
fmt.Printf("ret is %d\n", ret)
time.Sleep(time.Second * 4)
//释放module
slf.ReleaseModule(moduleId)
return false
}
func main() {
node := originnode.NewOriginNode()
if node == nil {
return
}
node.SetupService(&CTestService1{})
node.Init()
node.Start()
}
```
执行结果如下：
```
ret is 7
CTestModule1::OnInit
CTestModule1::OnRun
CTestModule1::OnRun
CTestModule1::OnRun
CTestModule1::OnRun
CTestModule1::OnEndRun
```
以上创建新的Module加入到当前服务对象中，可以获取释放动作。同样CTestModule1模块也可以加入子模块，使用方法一样。以上日志每秒钟CTestModule1::OnRun打印一次，4秒后ReleaseModule，对象被释放，执行CTestModule1::OnEndRun。
origin中其他重要服务:
---------------
* github.com\duanhf2012\origin\sysservice集成了系统常用的服务
* httpserverervice:提供对外的http服务
* wsserverservice :websocket服务
* logservice :日志服务
以上服务请参照github.com\duanhf2012\origin\Test目录使用方法
* github.com\duanhf2012\origin\sysmodule集成了系统常用的Module
* DBModule: mysql数据库操作模块，支持异步调用
* RedisModule: Redis操作模块，支持异步调用
* LogModule: 日志模块，支持区分日志等级
* HttpClientPoolModule:http客户端模块

21
Test/.vscode/launch.json vendored
View File

@@ -1,21 +0,0 @@
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "N_All",
"type": "go",
"request": "launch",
"mode": "auto",
"program": "${workspaceRoot}/main.go",
"env": {
"GOPATH":"${workspaceRoot}/../../../../../"
},
"args": ["NodeId=1"],
"output": "./OriginServer.exe"
}
]
}

3
Test/.vscode/settings.json vendored
View File

@@ -1,3 +0,0 @@
{
"go.gopath": "${workspaceRoot}/../../../../../"
}

4
Test/build.bat
View File

@@ -1,4 +0,0 @@
SET CGO_ENABLED=0
SET GOOS=linux
SET GOARCH=amd64
go build -v

55
Test/config/cluster.json
View File

@@ -1,55 +0,0 @@
{
"SubNet": [{
"Remark": "Manual,Full,Auto",
"SubNetMode": "Full",
"SubNetName": "SubNet1",
"PublicServiceList": ["logiclog"],
"NodeList": [{
"NodeID": 1,
"NodeName": "N_Node1",
"ServiceList": [
"HttpServerService",
"SubNet1_Service",
"TcpSocketPbService",
"ls"
],
"ClusterNode":["SubNet2.N_Node1","N_Node2"]
},
{
"NodeID": 2,
"NodeName": "N_Node2",
"ServiceList": [
"SubNet1_Service2"
],
"ClusterNode":[]
}
]
},
{
"Remark": "Manual,Full,Auto",
"SubNetMode": "Full",
"SubNetName": "SubNet2",
"PublicServiceList": ["logiclog"],
"NodeList": [{
"NodeID": 3,
"NodeName": "N_Node1",
"ServiceList": [
"SubNet2_Service1"
],
"ClusterNode":["URQ.N_Node1"]
}
]
}
]
}

50
Test/config/nodeconfig.json
View File

@@ -1,50 +0,0 @@
{
"Public":{
"LogLevel":1,
"HttpPort":9400,
"WSPort":9401,
"ListenPort":9412,
"CAFile":[
{
"CertFile":"",
"KeyFile":""
},
{
"CertFile":"",
"KeyFile":""
}
],
"Environment":"FS_Dev_LFY",
"IsListenLog":1,
"IsSendErrorMail":0
},
"NodeList":[
{
"NodeID":1,
"NodeAddr": "127.0.0.1:8181",
"LogLevel":1,
"HttpPort":7001,
"WSPort":7000,
"CertFile":"",
"KeyFile":""
},
{
"NodeID":2,
"NodeAddr": "127.0.0.1:8082"
}
,
{
"NodeID":3,
"NodeAddr": "127.0.0.1:8083"
}
,
{
"NodeID":40,
"NodeAddr": "127.0.0.1:8084",
"LogLevel":1
}
]
}

122
Test/logicservice/SubNet1_Service.go
View File

@@ -1,122 +0,0 @@
package logicservice
import (
"fmt"
"github.com/duanhf2012/origin/Test/msgpb"
"github.com/duanhf2012/origin/cluster"
"github.com/duanhf2012/origin/network"
"github.com/duanhf2012/origin/sysservice"
"github.com/golang/protobuf/proto"
"time"
"github.com/duanhf2012/origin/originnode"
"github.com/duanhf2012/origin/service"
"github.com/duanhf2012/origin/sysservice/originhttp"
)
type SubNet1_Service struct {
service.BaseService
}
func init() {
originnode.InitService(&SubNet1_Service{})
}
//OnInit ...
func (ws *SubNet1_Service) OnInit() error {
sysservice.GetTcpSocketPbService("ls").RegConnectEvent(ws.ConnEventHandler)
sysservice.GetTcpSocketPbService("ls").RegDisconnectEvent(ws.DisconnEventHandler)
sysservice.GetTcpSocketPbService("ls").RegExceptMessage(ws.ExceptMessage)
sysservice.GetTcpSocketPbService("ls").RegMessage(110, &msgpb.Test{}, ws.MessageHandler)
/*
sysservice.GetTcpSocketPbService("lc").RegConnectEvent(ws.ConnEventHandler2)
sysservice.GetTcpSocketPbService("lc").RegDisconnectEvent(ws.DisconnEventHandler2)
sysservice.GetTcpSocketPbService("lc").RegExceptMessage(ws.ExceptMessage2)
sysservice.GetTcpSocketPbService("lc").RegMessage(110, &msgpb.Test{}, ws.MessageHandler2)
*/
//originhttp.Post(" / aaa/bb/ :user/:pass/", ws.HTTP_UserIntegralInfo)
originhttp.Get(" /aaa/bbb", ws.HTTP_UserIntegralInfo)
originhttp.SetStaticResource(originhttp.METHOD_GET, "/file/", "d:\\")
return nil
}
type InputData struct {
A1 int
A2 int
}
//OnRun ...
func (ws *SubNet1_Service) OnRun() bool {
time.Sleep(time.Second * 10)
var cli network.TcpSocketClient
cli.Connect("127.0.0.1:9402")
test := msgpb.Test{}
test.AssistCount = proto.Int32(343)
cli.SendMsg(110, &test)
cli.SendMsg(110, &test)
var a InputData
var rs int
a.A1 = 3
a.A2 = 4
cluster.Call("SubNet1_Service2.RPC_Sub",&a,&rs)
fmt.Print(rs)
return false
}
func (ws *SubNet1_Service) MessageHandler(clientid uint64, msgtype uint16, msg proto.Message) {
fmt.Print("recv:",clientid, "：", msg,"\n")
sysservice.GetTcpSocketPbService("ls").SendMsg(clientid,msgtype,msg)
/*test core dump
var a map[int]int
a[33] = 3
fmt.Print(a[44])
*/
}
func (ws *SubNet1_Service) ConnEventHandler(clientid uint64) {
fmt.Print("connected..",clientid,"\n")
}
func (ws *SubNet1_Service) DisconnEventHandler(clientid uint64) {
fmt.Print("disconnected..",clientid,"\n")
}
func (ws *SubNet1_Service) ExceptMessage(clientid uint64, pPack *network.MsgBasePack, err error) {
fmt.Print("except..",clientid,"，",pPack,"\n")
}
///////////////////////////
func (ws *SubNet1_Service) MessageHandler2(clientid uint64, msgtype uint16, msg proto.Message) {
fmt.Print("recv:",clientid, "：", msg,"\n")
//pClient.SendMsg(msgtype,msg)
sysservice.GetTcpSocketPbService("ls").SendMsg(clientid,msgtype,msg)
}
func (ws *SubNet1_Service) ConnEventHandler2(clientid uint64) {
fmt.Print("connected..",clientid,"\n")
}
func (ws *SubNet1_Service) DisconnEventHandler2(clientid uint64) {
fmt.Print("disconnected..",clientid,"\n")
}
func (ws *SubNet1_Service) ExceptMessage2(clientid uint64, pPack *network.MsgBasePack, err error) {
fmt.Print("except..",clientid,"，",pPack,"\n")
}
func (slf *SubNet1_Service) HTTP_UserIntegralInfo(request *originhttp.HttpRequest, resp *originhttp.HttpRespone) error {
ret, ok := request.Query("a")
fmt.Print(ret, ok)
return nil
}

56
Test/main.go
View File

@@ -1,56 +0,0 @@
package main
import (
_ "github.com/duanhf2012/origin/Test/logicservice"
"github.com/duanhf2012/origin/cluster"
"github.com/duanhf2012/origin/network"
"github.com/duanhf2012/origin/originnode"
"github.com/duanhf2012/origin/sysservice"
"github.com/duanhf2012/origin/sysservice/originhttp"
"time"
)
type TcpSocketServerReciver struct {
}
func (slf *TcpSocketServerReciver) OnConnected(pClient *network.SClient){
}
func (slf *TcpSocketServerReciver) OnDisconnect(pClient *network.SClient){
}
func main() {
node := originnode.NewOriginNode()
if node == nil {
return
}
//打开Module死循环监控
node.EnableMonitorModule(time.Minute*5)
originhttp.SetStaticResource(originhttp.METHOD_GET,"/img/","d:/")
nodeCfg, _ := cluster.ReadNodeConfig("./config/nodeconfig.json", cluster.GetNodeId())
httpserver := originhttp.NewHttpServerService(nodeCfg.HttpPort) // http服务
for _, ca := range nodeCfg.CAFile {
httpserver.SetHttps(ca.CertFile, ca.KeyFile)
}
pTcpService := sysservice.NewTcpSocketPbService(":9412")
pTcpService.SetServiceName("ls")
/*
pTcpService2 := sysservice.NewTcpSocketPbService(":9005")
pTcpService2.SetServiceName("lc")
*/
httpserver.SetPrintRequestTime(true)
node.SetupService(httpserver,pTcpService)
node.Init()
node.Start()
}

1
Test/msgpb/gen.bat
View File

@@ -1 +0,0 @@
protoc --go_out=. test.proto

132
Test/msgpb/test.pb.go
View File

@@ -1,132 +0,0 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// source: test.proto
package msgpb
import (
fmt "fmt"
proto "github.com/golang/protobuf/proto"
math "math"
)
// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf
// This is a compile-time assertion to ensure that this generated file
// is compatible with the proto package it is being compiled against.
// A compilation error at this line likely means your copy of the
// proto package needs to be updated.
const _ = proto.ProtoPackageIsVersion2 // please upgrade the proto package
//*
// @brief base_score_info
type Test struct {
WinCount *int32 `protobuf:"varint,1,opt,name=win_count,json=winCount" json:"win_count,omitempty"`
LoseCount *int32 `protobuf:"varint,2,opt,name=lose_count,json=loseCount" json:"lose_count,omitempty"`
ExceptionCount *int32 `protobuf:"varint,3,opt,name=exception_count,json=exceptionCount" json:"exception_count,omitempty"`
KillCount *int32 `protobuf:"varint,4,opt,name=kill_count,json=killCount" json:"kill_count,omitempty"`
DeathCount *int32 `protobuf:"varint,5,opt,name=death_count,json=deathCount" json:"death_count,omitempty"`
AssistCount *int32 `protobuf:"varint,6,opt,name=assist_count,json=assistCount" json:"assist_count,omitempty"`
Rating *int64 `protobuf:"varint,7,opt,name=rating" json:"rating,omitempty"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
}
func (m *Test) Reset() { *m = Test{} }
func (m *Test) String() string { return proto.CompactTextString(m) }
func (*Test) ProtoMessage() {}
func (*Test) Descriptor() ([]byte, []int) {
return fileDescriptor_c161fcfdc0c3ff1e, []int{0}
}
func (m *Test) XXX_Unmarshal(b []byte) error {
return xxx_messageInfo_Test.Unmarshal(m, b)
}
func (m *Test) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
return xxx_messageInfo_Test.Marshal(b, m, deterministic)
}
func (m *Test) XXX_Merge(src proto.Message) {
xxx_messageInfo_Test.Merge(m, src)
}
func (m *Test) XXX_Size() int {
return xxx_messageInfo_Test.Size(m)
}
func (m *Test) XXX_DiscardUnknown() {
xxx_messageInfo_Test.DiscardUnknown(m)
}
var xxx_messageInfo_Test proto.InternalMessageInfo
func (m *Test) GetWinCount() int32 {
if m != nil && m.WinCount != nil {
return *m.WinCount
}
return 0
}
func (m *Test) GetLoseCount() int32 {
if m != nil && m.LoseCount != nil {
return *m.LoseCount
}
return 0
}
func (m *Test) GetExceptionCount() int32 {
if m != nil && m.ExceptionCount != nil {
return *m.ExceptionCount
}
return 0
}
func (m *Test) GetKillCount() int32 {
if m != nil && m.KillCount != nil {
return *m.KillCount
}
return 0
}
func (m *Test) GetDeathCount() int32 {
if m != nil && m.DeathCount != nil {
return *m.DeathCount
}
return 0
}
func (m *Test) GetAssistCount() int32 {
if m != nil && m.AssistCount != nil {
return *m.AssistCount
}
return 0
}
func (m *Test) GetRating() int64 {
if m != nil && m.Rating != nil {
return *m.Rating
}
return 0
}
func init() {
proto.RegisterType((*Test)(nil), "msgpb.test")
}
func init() { proto.RegisterFile("test.proto", fileDescriptor_c161fcfdc0c3ff1e) }
var fileDescriptor_c161fcfdc0c3ff1e = []byte{
// 178 bytes of a gzipped FileDescriptorProto
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0x3c, 0x8c, 0x5d, 0x0a, 0x82, 0x40,
0x14, 0x46, 0x99, 0xfc, 0x29, 0xaf, 0x51, 0xe0, 0x43, 0x08, 0x11, 0x59, 0x2f, 0xf9, 0xd4, 0x26,
0xda, 0x81, 0x1b, 0x08, 0xb3, 0xc1, 0x86, 0x6c, 0x46, 0xbc, 0x37, 0x6c, 0xc5, 0xad, 0x23, 0x66,
0xee, 0xe4, 0xe3, 0x77, 0xce, 0xe1, 0x03, 0x20, 0x89, 0x74, 0xee, 0x07, 0x43, 0x26, 0x8b, 0x5e,
0xd8, 0xf6, 0xb7, 0xe3, 0x57, 0x40, 0x68, 0x69, 0xb6, 0x85, 0x64, 0x54, 0xfa, 0xda, 0x98, 0xb7,
0xa6, 0x5c, 0x14, 0xa2, 0x8c, 0xaa, 0xc5, 0xa8, 0xf4, 0xc5, 0xee, 0x6c, 0x07, 0xd0, 0x19, 0x94,
0xde, 0xce, 0x9c, 0x4d, 0x2c, 0x61, 0x7d, 0x82, 0xb5, 0xfc, 0x34, 0xb2, 0x27, 0x65, 0xfe, 0x0f,
0x81, 0x6b, 0x56, 0x13, 0x9e, 0x7e, 0x9e, 0xaa, 0xeb, 0x7c, 0x13, 0xf2, 0x8f, 0x25, 0xac, 0xf7,
0x90, 0xde, 0x65, 0x4d, 0x0f, 0xef, 0x23, 0xe7, 0xc1, 0x21, 0x0e, 0x0e, 0xb0, 0xac, 0x11, 0x15,
0x92, 0x2f, 0x62, 0x57, 0xa4, 0xcc, 0x38, 0xd9, 0x40, 0x3c, 0xd4, 0xa4, 0x74, 0x9b, 0xcf, 0x0b,
0x51, 0x06, 0x95, 0x5f, 0xbf, 0x00, 0x00, 0x00, 0xff, 0xff, 0x46, 0xa0, 0x89, 0x59, 0xfc, 0x00,
0x00, 0x00,
}

18
Test/msgpb/test.proto
View File

@@ -1,18 +0,0 @@
syntax = "proto2";
package msgpb;
/**
* @brief base_score_info
*/
message test{
optional int32 win_count = 1; // 玩家胜局局数
optional int32 lose_count = 2; // 玩家负局局数
optional int32 exception_count = 3; // 玩家异常局局数
optional int32 kill_count = 4; // 总人头数
optional int32 death_count = 5; // 总死亡数
optional int32 assist_count = 6; // 总总助攻数
optional int64 rating = 7; // 评价积分
}

13
Test/workspace.code-workspace
View File

@@ -1,13 +0,0 @@
{
"folders": [
{
"path": "."
},
{
"path": "D:\\GOPATH\\src\\github.com"
}
],
"settings": {
"http.systemCertificates": false
}
}

21
Test2/Helloworld/.vscode/launch.json vendored
View File

@@ -1,21 +0,0 @@
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "N_All",
"type": "go",
"request": "launch",
"mode": "auto",
"program": "${workspaceRoot}/main.go",
"env": {
"GOPATH":"${workspaceRoot}/../../../../../../"
},
"args": ["NodeId=1"],
"output": "./OriginServer.exe"
}
]
}

3
Test2/Helloworld/.vscode/settings.json vendored
View File

@@ -1,3 +0,0 @@
{
"go.gopath": "${workspaceRoot}/../../../../../"
}

4
Test2/Helloworld/build.bat
View File

@@ -1,4 +0,0 @@
SET CGO_ENABLED=0
SET GOOS=linux
SET GOARCH=amd64
go build -v

37
Test2/Helloworld/config/cluster.json
View File

@@ -1,37 +0,0 @@
{
"SubNet": [{
"Remark": "Manual,Full,Auto",
"SubNetMode": "Full",
"SubNetName": "SubNet1",
"PublicServiceList": ["logiclog"],
"NodeList": [{
"NodeID": 1,
"NodeName": "N_Node1",
"ServiceList": [
"TestService1",
"TestService2",
"HttpServerService"
],
"ClusterNode":[]
},
{
"NodeID": 2,
"NodeName": "N_Node2",
"ServiceList": [
"TestService3"
],
"ClusterNode":[]
}
]
}
]
}

20
Test2/Helloworld/config/nodeconfig.json
View File

@@ -1,20 +0,0 @@
{
"Public": {
"LogLevel": 1,
"HttpPort": 9400,
"WSPort": 9500,
"Environment": "Test",
"IsListenLog": 1,
"IsSendErrorMail": 0
},
"NodeList": [{
"NodeID": 1,
"NodeAddr": "127.0.0.1:8081"
},
{
"NodeID": 2,
"NodeAddr": "127.0.0.1:8082"
}
]
}

16
Test2/Helloworld/main.go
View File

@@ -1,16 +0,0 @@
package main
import (
"github.com/duanhf2012/origin/originnode"
)
func main() {
node := originnode.NewOriginNode()
if node == nil {
return
}
node.Init()
node.Start()
}

10
Test2/Helloworld/workspace.code-workspace
View File

@@ -1,10 +0,0 @@
{
"folders": [
{
"path": "."
}
],
"settings": {
"http.systemCertificates": false
}
}

1
_config.yml
View File

@@ -1 +0,0 @@
theme: jekyll-theme-leap-day

786
cluster/cluster.go
View File

@@ -1,723 +1,135 @@
package cluster
import (
"errors"
"fmt"
"math/rand"
"net"
"os"
"sort"
"github.com/duanhf2012/originnet/log"
"github.com/duanhf2012/originnet/rpc"
"github.com/duanhf2012/originnet/service"
"strings"
"time"
"github.com/duanhf2012/origin/rpc"
"github.com/duanhf2012/origin/service"
"github.com/duanhf2012/origin/sysmodule"
)
type RpcClient struct {
nodeid int
pclient *rpc.Client
serverAddr string
var configdir = "./config/"
type SubNet struct {
NodeList []NodeInfo
}
func (slf *RpcClient) IsConnected() bool {
return (slf.pclient != nil) && (slf.pclient.IsClosed() == false)
type NodeInfo struct {
NodeId int
ListenAddr string
NodeName string
ServiceList []string
}
type CCluster struct {
port int
cfg *ClusterConfig
nodeclient map[int]*RpcClient
reader net.Conn
writer net.Conn
LocalRpcClient *rpc.Client
innerLocalServiceList map[string]bool
type NodeRpcInfo struct {
nodeinfo NodeInfo
client *rpc.Client
}
func (slf *CCluster) ReadNodeInfo(nodeid int) error {
mapNodeData, err := ReadAllNodeConfig("./config/nodeconfig.json")
var cluster Cluster
type Cluster struct {
localsubnet SubNet //本子网
mapSubNetInfo map[string] SubNet //子网名称，子网信息
mapSubNetNodeInfo map[string]map[int]NodeInfo //map[子网名称]map[NodeId]NodeInfo
localSubNetMapNode map[int]NodeInfo //本子网内 map[NodeId]NodeInfo
localSubNetMapService map[string][]NodeInfo //本子网内所有ServiceName对应的结点列表
localNodeMapService map[string]interface{} //本Node支持的服务
localNodeInfo NodeInfo
localNodeServiceCfg map[string]interface{} //map[servicename]数据
mapRpc map[int] NodeRpcInfo//nodeid
rpcServer rpc.Server
}
func SetConfigDir(cfgdir string){
configdir = cfgdir
}
func (slf *Cluster) Init(currentNodeId int) error{
//1.初始化配置
err := slf.InitCfg(currentNodeId)
if err != nil {
return err
}
slf.cfg, err = ReadCfg("./config/cluster.json", nodeid, mapNodeData)
if err != nil {
return err
slf.rpcServer.Init(slf)
//2.建议rpc连接
slf.mapRpc = map[int] NodeRpcInfo{}
for _,nodeinfo := range slf.localSubNetMapNode {
rpcinfo := NodeRpcInfo{}
rpcinfo.nodeinfo = nodeinfo
rpcinfo.client = &rpc.Client{}
if nodeinfo.NodeId == currentNodeId {
rpcinfo.client.Connect("localhost")
//rpcinfo.client.Connect(nodeinfo.ListenAddr)
}else{
rpcinfo.client.Connect(nodeinfo.ListenAddr)
}
slf.mapRpc[nodeinfo.NodeId] = rpcinfo
}
return nil
}
func (slf *CCluster) GetClusterClient(id int) *rpc.Client {
if id == GetNodeId() {
return slf.LocalRpcClient
func (slf *Cluster) FindRpcHandler(servicename string) rpc.IRpcHandler {
pService := service.GetService(servicename)
if pService == nil {
return nil
}
v, ok := slf.nodeclient[id]
return pService.GetRpcHandler()
}
func (slf *Cluster) Start() {
slf.rpcServer.Start(slf.localNodeInfo.ListenAddr)
}
func GetCluster() *Cluster{
return &cluster
}
func (slf *Cluster) GetRpcClient(nodeid int) *rpc.Client {
c,ok := slf.mapRpc[nodeid]
if ok == false {
return nil
}
return v.pclient
return c.client
}
func (slf *CCluster) GetBindUrl() string {
return slf.cfg.currentNode.ServerAddr
}
func GetRpcClient(serviceMethod string) ([]*rpc.Client,error) {
serviceAndMethod := strings.Split(serviceMethod,".")
if len(serviceAndMethod)!=2 {
return nil,fmt.Errorf("servicemethod param %s is error!",serviceMethod)
}
func (slf *CCluster) AcceptRpc(tpcListen *net.TCPListener) error {
for {
conn, err := tpcListen.Accept()
if err != nil {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "tpcListen.Accept error:%v", err)
return err
//1.找到对应的rpcnodeid
var rpcClientList []*rpc.Client
nodeidList := GetCluster().GetNodeIdByService(serviceAndMethod[0])
if len(nodeidList) ==0 {
return rpcClientList,fmt.Errorf("Cannot Find %s nodeid",serviceMethod)
}
for _,nodeid:= range nodeidList {
pClient := GetCluster().GetRpcClient(nodeid)
if pClient==nil {
log.Error("Cannot connect node id %d",nodeid)
continue
}
go rpc.ServeConn(conn)
rpcClientList = append(rpcClientList,pClient)
}
return nil
return rpcClientList,nil
}
func (slf *CCluster) ListenService() error {
bindStr := slf.GetBindUrl()
parts := strings.Split(bindStr, ":")
if len(parts) < 2 {
service.GetLogger().Printf(sysmodule.LEVER_FATAL, "ListenService address %s is error.", bindStr)
os.Exit(1)
}
bindStr = "0.0.0.0:" + parts[1]
//
tcpaddr, err := net.ResolveTCPAddr("tcp4", bindStr)
if err != nil {
service.GetLogger().Printf(sysmodule.LEVER_FATAL, "ResolveTCPAddr error:%v", err)
os.Exit(1)
return err
}
tcplisten, err2 := net.ListenTCP("tcp", tcpaddr)
if err2 != nil {
service.GetLogger().Printf(sysmodule.LEVER_FATAL, "ListenTCP error:%v", err2)
os.Exit(1)
return err2
}
go slf.AcceptRpc(tcplisten)
slf.ReSetLocalRpcClient()
return nil
}
func (slf *CCluster) ReSetLocalRpcClient() {
slf.reader, slf.writer = net.Pipe()
go rpc.ServeConn(slf.reader)
slf.LocalRpcClient = rpc.NewClient(slf.writer,true)
}
type CPing struct {
TimeStamp int64
}
type CPong struct {
TimeStamp int64
}
func (slf *CPing) Ping(ping *CPing, pong *CPong) error {
pong.TimeStamp = ping.TimeStamp
return nil
}
func (slf *CCluster) GetClusterMode() string {
return slf.cfg.GetClusterMode()
}
func (slf *CCluster) ConnService() error {
ping := CPing{0}
pong := CPong{0}
rpc.RegisterName("CPing", "", &ping)
//连接集群服务器
for _, nodeList := range slf.cfg.mapClusterNodeService {
for _, node := range nodeList {
if node.NodeID == slf.cfg.currentNode.NodeID {
continue
}
slf.nodeclient[node.NodeID] = &RpcClient{node.NodeID, nil, node.ServerAddr}
}
}
//判断集群模式
for {
for _, rpcClient := range slf.nodeclient {
//连接状态发送ping
if rpcClient.IsConnected() == true {
ping.TimeStamp = 0
err := rpcClient.pclient.Call("CPing.Ping", &ping, &pong)
if err != nil {
rpcClient.pclient.Close()
rpcClient.pclient = nil
continue
}
continue
}
//非连接状态重新连接
if rpcClient.pclient != nil {
rpcClient.pclient.Close()
rpcClient.pclient = nil
}
client, err := rpc.Dial("tcp", rpcClient.serverAddr)
if err != nil {
service.GetLogger().Printf(sysmodule.LEVER_WARN, "Connect nodeid:%d,address:%s fail", rpcClient.nodeid, rpcClient.serverAddr)
continue
}
service.GetLogger().Printf(sysmodule.LEVER_INFO, "Connect nodeid:%d,address:%s succ", rpcClient.nodeid, rpcClient.serverAddr)
v, _ := slf.nodeclient[rpcClient.nodeid]
v.pclient = client
}
if slf.LocalRpcClient.IsClosed() {
slf.ReSetLocalRpcClient()
}
time.Sleep(time.Second * 4)
}
return nil
}
func (slf *CCluster) Init(currentNodeid int) error {
slf.nodeclient = make(map[int]*RpcClient)
return slf.ReadNodeInfo(currentNodeid)
}
func (slf *CCluster) Start() error {
service.InstanceServiceMgr().FetchService(slf.OnFetchService)
//监听服务
slf.ListenService()
//集群
go slf.ConnService()
return nil
}
//servicename.methodname
//_servicename.methodname
func (slf *CCluster) Call(NodeServiceMethod string, args interface{}, reply interface{}) error {
var callServiceName string
var serviceName string
nodeidList := slf.GetNodeList(NodeServiceMethod, &callServiceName, &serviceName)
if len(nodeidList) > 1 || len(nodeidList) < 1 {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "CCluster.Call(%s) find nodes count %d is error.", NodeServiceMethod, len(nodeidList))
return fmt.Errorf("CCluster.Call(%s) find nodes count %d is error.", NodeServiceMethod, len(nodeidList))
}
nodeid := nodeidList[0]
if nodeid == GetNodeId() {
//判断服务是否已经完成初始化
iService := service.InstanceServiceMgr().FindService(serviceName)
if iService == nil {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "CCluster.Call(%s): NodeId %d cannot find service.", NodeServiceMethod, nodeid)
return fmt.Errorf("CCluster.Call(%s): NodeId %d cannot find service..", NodeServiceMethod, nodeid)
}
if iService.IsInit() == false {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "CCluster.Call(%s): NodeId %d is not init.", NodeServiceMethod, nodeid)
return fmt.Errorf("CCluster.Call(%s): NodeId %d is not init.", NodeServiceMethod, nodeid)
}
return slf.LocalRpcClient.Call(callServiceName, args, reply)
} else {
pclient := slf.GetClusterClient(nodeid)
if pclient == nil {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "CCluster.Call(%s): NodeId %d is not find.", NodeServiceMethod, nodeid)
return fmt.Errorf("CCluster.Call(%s): NodeId %d is not find.", NodeServiceMethod, nodeid)
}
err := pclient.Call(callServiceName, args, reply)
if err != nil {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "CCluster.Call(%s) is fail:%v.", callServiceName, err)
}
return err
}
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "CCluster.Call(%s) fail.", NodeServiceMethod)
return fmt.Errorf("CCluster.Call(%s) fail.", NodeServiceMethod)
}
func (slf *CCluster) GetNodeList(NodeServiceMethod string, rpcServerMethod *string, rpcServiceName *string) []int {
var nodename string
var servicename string
var methodname string
var nodeidList []int
parts := strings.Split(NodeServiceMethod, ".")
if len(parts) == 2 {
servicename = parts[0]
methodname = parts[1]
} else if len(parts) == 3 {
nodename = parts[0]
servicename = parts[1]
methodname = parts[2]
} else {
return nodeidList
}
if nodename == "" {
nodeidList = make([]int, 0)
if servicename[:1] == "_" {
servicename = servicename[1:]
nodeidList = append(nodeidList, GetNodeId())
} else {
nodeidList = slf.cfg.GetIdByService(servicename, "")
}
} else {
nodeidList = slf.GetIdByNodeService(nodename, servicename)
}
if rpcServiceName != nil {
*rpcServiceName = servicename
}
if rpcServerMethod != nil {
*rpcServerMethod = servicename + "." + methodname
}
return nodeidList
}
//GetNodeIdByServiceName 根据服务名查找nodeid servicename服务名 bOnline是否需要查找在线服务
func (slf *CCluster) GetNodeIdByServiceName(servicename string, bOnline bool) []int {
nodeIDList := slf.cfg.GetIdByService(servicename, "")
if bOnline {
ret := make([]int, 0, len(nodeIDList))
for _, nodeid := range nodeIDList {
if slf.CheckNodeIsConnectedByID(nodeid) {
ret = append(ret, nodeid)
}
}
return ret
}
return nodeIDList
}
//根据Service获取负载均衡信息
//负载均衡的策略是从配置获取所有配置了该服务的NodeId 并按NodeId排序 每个node负责处理数组index所在的那一部分
func (slf *CCluster) GetBalancingInfo(currentNodeId int, servicename string, inSubNet bool) (*BalancingInfo, error) {
subNetName := ""
if inSubNet {
if node, ok := slf.cfg.mapIdNode[currentNodeId]; ok {
subNetName = node.SubNetName
} else {
return nil, fmt.Errorf("[cluster.GetBalancingInfo] cannot find node %d", currentNodeId)
}
}
lst := slf.cfg.GetIdByService(servicename, subNetName)
// if len(lst) <= 0 {
// return nil, fmt.Errorf("[cluster.GetBalancingInfo] cannot find service %s in any node", servicename)
// }
sort.Ints(lst)
ret := &BalancingInfo{
NodeId: currentNodeId,
ServiceName: servicename,
TotalNum: len(lst),
MyIndex: -1,
NodeList: lst,
}
if _, ok := slf.cfg.mapIdNode[currentNodeId]; ok {
for i, v := range lst {
if v == currentNodeId {
ret.MyIndex = i
break
}
}
}
return ret, nil
}
func (slf *CCluster) CheckNodeIsConnectedByID(nodeid int) bool {
if nodeid == GetNodeId() {
return true
}
pclient := slf.GetRpcClientByNodeId(nodeid)
if pclient == nil {
return false
}
return pclient.IsConnected()
}
func (slf *CCluster) GetRpcClientByNodeId(nodeid int) *RpcClient {
pclient, ok := slf.nodeclient[nodeid]
if ok == false {
return nil
}
return pclient
}
func (slf *CCluster) Go(bCast bool, NodeServiceMethod string, args interface{}, queueModle bool) error {
return slf.goImpl(bCast, NodeServiceMethod, args, queueModle, true)
}
func (slf *CCluster) goImpl(bCast bool, NodeServiceMethod string, args interface{}, queueModle bool, log bool) error {
var callServiceName string
var serviceName string
nodeidList := slf.GetNodeList(NodeServiceMethod, &callServiceName, &serviceName)
if len(nodeidList) < 1 {
err := fmt.Errorf("CCluster.Go(%s) not find nodes.", NodeServiceMethod)
if log {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, err.Error())
}
return err
}
if bCast == false && len(nodeidList) > 1 {
return fmt.Errorf("CCluster.Go(%s) find more nodes", NodeServiceMethod)
}
for _, nodeid := range nodeidList {
if nodeid == GetNodeId() {
iService := service.InstanceServiceMgr().FindService(serviceName)
if iService == nil {
return fmt.Errorf("CCluster.Go(%s) cannot find service %s", NodeServiceMethod, serviceName)
}
if iService.IsInit() == false {
err := fmt.Errorf("CCluster.Call(%s): NodeId %d is not init.", NodeServiceMethod, nodeid)
if log {
service.GetLogger().Printf(sysmodule.LEVER_WARN, err.Error())
}
return err
}
replyCall := slf.LocalRpcClient.Go(callServiceName, args, nil, nil, queueModle)
if replyCall.Error != nil {
err := fmt.Errorf("CCluster.Go(%s) fail:%v.", NodeServiceMethod, replyCall.Error)
if log {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, err.Error())
} else {
return err
}
}
} else {
pclient := slf.GetClusterClient(nodeid)
if pclient == nil {
err := fmt.Errorf("CCluster.Go(%s) NodeId %d not find client", NodeServiceMethod, nodeid)
if log {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, err.Error())
}
return err
}
replyCall := pclient.Go(callServiceName, args, nil, nil, queueModle)
if replyCall.Error != nil {
err := fmt.Errorf("CCluster.Go(%s) fail:%v.", NodeServiceMethod, replyCall.Error)
if log {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, err.Error())
}
return err
}
}
}
return nil
}
func (slf *CCluster) CallNode(nodeid int, servicemethod string, args interface{}, reply interface{}) error {
pclient := slf.GetClusterClient(nodeid)
if pclient == nil {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "CCluster.CallNode(%d,%s) NodeId not find client", nodeid, servicemethod)
return fmt.Errorf("Call: NodeId %d is not find.", nodeid)
}
err := pclient.Call(servicemethod, args, reply)
if err != nil {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "CCluster.CallNode(%d,%s) fail:%v", nodeid, servicemethod, err)
}
return err
}
func (slf *CCluster) GoNode(nodeid int, args interface{}, servicemethod string, queueModle bool) error {
pclient := slf.GetClusterClient(nodeid)
if pclient == nil {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "CCluster.GoNode(%d,%s) NodeId not find client", nodeid, servicemethod)
return fmt.Errorf("CCluster.GoNode(%d,%s) NodeId not find client", nodeid, servicemethod)
}
replyCall := pclient.Go(servicemethod, args, nil, nil, queueModle)
if replyCall.Error != nil {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "CCluster.GoNode(%d,%s) fail:%v", nodeid, servicemethod, replyCall.Error)
}
return replyCall.Error
}
func (ws *CCluster) OnFetchService(iservice service.IService) error {
rpc.RegisterName(iservice.GetServiceName(), "RPC_", iservice)
return nil
}
func (slf *CCluster) CallRandomService(NodeServiceMethod string, args interface{}, reply interface{}) error {
var servicename string
parts := strings.Split(NodeServiceMethod, ".")
if len(parts) == 2 {
servicename = parts[0]
} else {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "CCluster.CallRandomService(%s) method err", NodeServiceMethod)
return fmt.Errorf("CCluster.GoNode(%s) NodeId method err", NodeServiceMethod)
}
nodeList := slf.GetNodeIdByServiceName(servicename, true)
if len(nodeList) < 1 {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "CCluster.CallRandomService(%s) no node is online", NodeServiceMethod)
return fmt.Errorf("CCluster.GoNode(%s) no node is online", NodeServiceMethod)
}
nodeIndex := rand.Intn(len(nodeList))
nodeID := nodeList[nodeIndex]
return slf.CallNode(nodeID, NodeServiceMethod, args, reply)
}
func (slf *CCluster) GoRandomService(NodeServiceMethod string, args interface{}, queueModle bool) error {
var servicename string
parts := strings.Split(NodeServiceMethod, ".")
if len(parts) == 2 {
servicename = parts[0]
} else {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "CCluster.CallRandomService(%s) method err", NodeServiceMethod)
return fmt.Errorf("CCluster.GoNode(%s) NodeId method err", NodeServiceMethod)
}
nodeList := slf.GetNodeIdByServiceName(servicename, true)
if len(nodeList) < 1 {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "CCluster.CallRandomService(%s) no node is online", NodeServiceMethod)
return fmt.Errorf("CCluster.GoNode(%s) no node is online", NodeServiceMethod)
}
nodeIndex := rand.Intn(len(nodeList))
nodeID := nodeList[nodeIndex]
return slf.GoNode(nodeID, args, NodeServiceMethod, queueModle)
}
//向远程服务器调用
//Node.servicename.methodname
//servicename.methodname
func Call(NodeServiceMethod string, args interface{}, reply interface{}) error {
return InstanceClusterMgr().Call(NodeServiceMethod, args, reply)
}
func (slf *CCluster) CallEx(NodeServiceMethod string, args interface{}, reply interface{}) *RpcCallResult {
return slf.rawcall(NodeServiceMethod, args, reply, false)
}
type RpcCallResult struct {
chanRet chan *rpc.Call
err error
rets *rpc.Call
}
func (slf *RpcCallResult) Make() {
slf.chanRet = make(chan *rpc.Call, 1)
slf.rets = nil
}
func (slf *RpcCallResult) WaitReturn(waittm time.Duration) error {
if slf.chanRet == nil {
return errors.New("cannot make rpccallresult")
}
if waittm <= 0 {
select {
case ret := <-slf.chanRet:
return ret.Error
}
} else {
//
select {
case ret := <-slf.chanRet:
return ret.Error
case <-time.After(waittm):
return errors.New("is time out")
}
}
return errors.New("unknow error.")
}
func (slf *CCluster) rawcall(NodeServiceMethod string, args interface{}, reply interface{}, queueModle bool) *RpcCallResult {
var rpcRet RpcCallResult
rpcRet.Make()
var callServiceName string
var serviceName string
nodeidList := slf.GetNodeList(NodeServiceMethod, &callServiceName, &serviceName)
if len(nodeidList) > 1 || len(nodeidList) < 1 {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "CCluster.Call(%s) find nodes count %d is error.", NodeServiceMethod, len(nodeidList))
rpcRet.err = fmt.Errorf("CCluster.Call(%s) find nodes count %d is error.", NodeServiceMethod, len(nodeidList))
return &rpcRet
}
nodeid := nodeidList[0]
if nodeid == GetNodeId() {
//判断服务是否已经完成初始化
iService := service.InstanceServiceMgr().FindService(serviceName)
if iService == nil {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "CCluster.Call(%s): NodeId %d cannot find service.", NodeServiceMethod, nodeid)
rpcRet.err = fmt.Errorf("CCluster.Call(%s): NodeId %d cannot find service..", NodeServiceMethod, nodeid)
return &rpcRet
}
if iService.IsInit() == false {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "CCluster.Call(%s): NodeId %d is not init.", NodeServiceMethod, nodeid)
rpcRet.err = fmt.Errorf("CCluster.Call(%s): NodeId %d is not init.", NodeServiceMethod, nodeid)
return &rpcRet
}
rpcRet.rets = slf.LocalRpcClient.Go(callServiceName, args, reply, rpcRet.chanRet, queueModle)
return &rpcRet
}
pclient := slf.GetClusterClient(nodeid)
if pclient == nil {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "CCluster.Call(%s): NodeId %d is not find.", NodeServiceMethod, nodeid)
rpcRet.err = fmt.Errorf("CCluster.Call(%s): NodeId %d is not find.", NodeServiceMethod, nodeid)
return &rpcRet
}
rpcRet.rets = pclient.Go(callServiceName, args, reply, rpcRet.chanRet, queueModle)
return &rpcRet
}
func CallEx(NodeServiceMethod string, args interface{}, reply interface{}) *RpcCallResult {
return InstanceClusterMgr().rawcall(NodeServiceMethod, args, reply, false)
}
func CallNode(NodeId int, servicemethod string, args interface{}, reply interface{}) error {
return InstanceClusterMgr().CallNode(NodeId, servicemethod, args, reply)
}
func GoNode(NodeId int, servicemethod string, args interface{}) error {
return InstanceClusterMgr().GoNode(NodeId, args, servicemethod, false)
}
func Go(NodeServiceMethod string, args interface{}) error {
return InstanceClusterMgr().Go(false, NodeServiceMethod, args, false)
}
func CastGo(NodeServiceMethod string, args interface{}) error {
return InstanceClusterMgr().Go(true, NodeServiceMethod, args, false)
}
func GoNodeQueue(NodeId int, servicemethod string, args interface{}) error {
return InstanceClusterMgr().GoNode(NodeId, args, servicemethod, true)
}
func GoQueue(NodeServiceMethod string, args interface{}) error {
return InstanceClusterMgr().Go(false, NodeServiceMethod, args, true)
}
//在GoQueue的基础上增加是否写日志参数
func GoQueueEx(NodeServiceMethod string, args interface{}, log bool) error {
return InstanceClusterMgr().goImpl(false, NodeServiceMethod, args, true, log)
}
func CastGoQueue(NodeServiceMethod string, args interface{}) error {
return InstanceClusterMgr().Go(true, NodeServiceMethod, args, true)
}
//GetNodeIdByServiceName 根据服务名查找nodeid serviceName服务名 bOnline是否需要查找在线服务
func GetNodeIdByServiceName(serviceName string, bOnline bool) []int {
return InstanceClusterMgr().GetNodeIdByServiceName(serviceName, bOnline)
}
//获取服务的负载均衡信息
//负载均衡的策略是从配置获取所有配置了该服务的NodeId 并按NodeId排序 每个node负责处理数组index所在的那一部分
func GetBalancingInfo(currentNodeId int, servicename string, inSubNet bool) (*BalancingInfo, error) {
return InstanceClusterMgr().GetBalancingInfo(currentNodeId, servicename, inSubNet)
}
//随机选择在线的node发送
func CallRandomService(NodeServiceMethod string, args interface{}, reply interface{}) error {
return InstanceClusterMgr().CallRandomService(NodeServiceMethod, args, reply)
}
func GoRandomService(NodeServiceMethod string, args interface{}) error {
return InstanceClusterMgr().GoRandomService(NodeServiceMethod, args, false)
}
func GoRandomServiceQueue(NodeServiceMethod string, args interface{}) error {
return InstanceClusterMgr().GoRandomService(NodeServiceMethod, args, true)
}
var _self *CCluster
func InstanceClusterMgr() *CCluster {
if _self == nil {
_self = new(CCluster)
_self.innerLocalServiceList = make(map[string]bool)
return _self
}
return _self
}
func (slf *CCluster) GetIdByNodeService(NodeName string, serviceName string) []int {
return slf.cfg.GetIdByNodeService(NodeName, serviceName)
}
func (slf *CCluster) HasLocalService(serviceName string) bool {
_, ok := slf.innerLocalServiceList[serviceName]
return slf.cfg.HasLocalService(serviceName) || ok
}
func (slf *CCluster) HasInit(serviceName string) bool {
return slf.cfg.HasLocalService(serviceName)
}
func GetNodeId() int {
return _self.cfg.currentNode.NodeID
}
func (slf *CCluster) AddLocalService(iservice service.IService) {
servicename := fmt.Sprintf("%T", iservice)
parts := strings.Split(servicename, ".")
if len(parts) != 2 {
service.GetLogger().Printf(service.LEVER_ERROR, "BaseService.Init: service name is error: %q", servicename)
}
servicename = parts[1]
slf.innerLocalServiceList[servicename] = true
}
func GetNodeName(nodeid int) string {
return _self.cfg.GetNodeNameByNodeId(nodeid)
}
func DynamicCall(address string, serviceMethod string, args interface{}, reply interface{}) error {
rpcClient, err := rpc.DialTimeOut("tcp", address, time.Second*1)
if err != nil {
return err
}
defer rpcClient.Close()
err = rpcClient.Call(serviceMethod, args, reply)
if err != nil {
return err
}
return nil
}
func GetRpcServer() *rpc.Server{
return &cluster.rpcServer
}

341
cluster/config.go
View File

@@ -1,341 +0,0 @@
package cluster
import (
"encoding/json"
"errors"
"fmt"
"io/ioutil"
"strings"
)
//负载均衡信息
type BalancingInfo struct {
NodeId int //我的nodeId
ServiceName string //负载均衡的ServiceName
TotalNum int //总共有多少个协同Node
MyIndex int //负责的index [0, TotalNum)
NodeList []int //所有协同的node列表 按NodeId升序排列
}
//判断hash后的Id是否命中我的NodeId
func (slf *BalancingInfo) Hit(hashId int) bool {
if hashId >= 0 && slf.TotalNum > 0 && slf.MyIndex >= 0 {
return hashId%slf.TotalNum == slf.MyIndex
}
return false
}
//判断命中的NodeId，-1表示无法取得
func (slf *BalancingInfo) GetHitNodeId(hashId int) int {
if hashId >= 0 && slf.TotalNum > 0 {
if idx := hashId % slf.TotalNum; idx >= 0 && idx < len(slf.NodeList) {
return slf.NodeList[idx]
}
}
return -1
}
type CNodeCfg struct {
NodeID int
NodeName string
ServiceList []string
ClusterNode []string
}
type CNode struct {
SubNetName string
NodeID int
NodeName string //SubNetName.NodeName
ServerAddr string
ServiceList map[string]bool
}
type SubNetNodeInfo struct {
SubNetMode string
SubNetName string
PublicServiceList []string
NodeList []CNodeCfg //配置列表
//mapClusterNodeService map[string][]CNode //map[nodename] []CNode
//mapClusterServiceNode map[string][]CNode //map[servicename] []CNode
}
type ClusterConfig struct {
SubNet []SubNetNodeInfo
//CurrentSubNetIdx int
mapIdNode map[int]CNode
currentNode CNode //当前node
mapClusterNodeService map[string][]CNode //map[nodename] []CNode
mapClusterServiceNode map[string][]CNode //map[servicename] []CNode
}
func GenNodeName(subnetName string, nodename string) string {
parts := strings.Split(nodename, ".")
if len(parts) < 2 {
return subnetName + "." + nodename
}
return nodename
}
func AddCluster(clusterNodeNameList *[]string, nodename string) bool {
for _, n := range *clusterNodeNameList {
if n == nodename {
return false
}
}
*clusterNodeNameList = append(*clusterNodeNameList, nodename)
return true
}
func ReadCfg(path string, nodeid int, mapNodeData map[int]NodeData) (*ClusterConfig, error) {
clsCfg := &ClusterConfig{}
clsCfg.mapIdNode = map[int]CNode{}
clsCfg.mapClusterNodeService = make(map[string][]CNode, 1)
clsCfg.mapClusterServiceNode = make(map[string][]CNode, 1)
//1.加载解析配置
d, err := ioutil.ReadFile(path)
if err != nil {
fmt.Printf("Read File %s Error!", path)
return nil, err
}
err = json.Unmarshal(d, clsCfg)
if err != nil {
fmt.Printf("Read File %s ,%s Error!", path, err)
return nil, err
}
//存储所有的nodeid对应cnode信息
var clusterNodeNameList []string
for _, c := range clsCfg.SubNet {
for _, v := range c.NodeList {
mapservice := make(map[string]bool, 1)
for _, s := range v.ServiceList {
mapservice[s] = true
}
nodeData, ok := mapNodeData[v.NodeID]
if ok == false {
return nil, errors.New(fmt.Sprintf("Cannot find node id %d in nodeconfig.json file!", v.NodeID))
}
node := CNode{c.SubNetName, v.NodeID, c.SubNetName + "." + v.NodeName, nodeData.NodeAddr, mapservice}
clsCfg.mapIdNode[v.NodeID] = node
if v.NodeID == nodeid {
clsCfg.currentNode = node
for _, servicename := range c.PublicServiceList {
clsCfg.currentNode.ServiceList[servicename] = true
}
for _, nodename := range v.ClusterNode {
AddCluster(&clusterNodeNameList, GenNodeName(c.SubNetName, nodename))
}
AddCluster(&clusterNodeNameList, GenNodeName(c.SubNetName, v.NodeName))
}
}
}
if clsCfg.currentNode.NodeID == 0 {
return nil, errors.New(fmt.Sprintf("Cannot find NodeId %d in cluster.json!", nodeid))
}
//如果集群是FULL模式
if strings.ToUpper(clsCfg.GetClusterMode()) == "FULL" {
for _, subnet := range clsCfg.SubNet {
if subnet.SubNetName == clsCfg.currentNode.SubNetName {
for _, nodes := range subnet.NodeList {
AddCluster(&clusterNodeNameList, GenNodeName(subnet.SubNetName, nodes.NodeName))
}
}
}
}
for _, clusternodename := range clusterNodeNameList {
for _, c := range clsCfg.SubNet {
for _, nodecfg := range c.NodeList {
if clusternodename != c.SubNetName+"."+nodecfg.NodeName {
continue
}
n, ok := clsCfg.mapIdNode[nodecfg.NodeID]
if ok == false {
return nil, errors.New(fmt.Sprintf("Cannot find NodeId %d in cluster.json!", nodecfg.NodeID))
}
clsCfg.mapClusterNodeService[nodecfg.NodeName] = append(clsCfg.mapClusterNodeService[nodecfg.NodeName], n)
for _, sname := range nodecfg.ServiceList {
clsCfg.mapClusterServiceNode[sname] = append(clsCfg.mapClusterServiceNode[sname], n)
}
}
}
}
return clsCfg, nil
}
func (slf *ClusterConfig) GetIdByService(serviceName, subNetName string) []int {
var nodeidlist = []int{}
nodeList, ok := slf.mapClusterServiceNode[serviceName]
if ok == true {
nodeidlist = make([]int, 0, len(nodeList))
for _, v := range nodeList {
if subNetName == "" || subNetName == v.SubNetName {
nodeidlist = append(nodeidlist, v.NodeID)
}
}
}
return nodeidlist
}
func (slf *ClusterConfig) GetIdByNodeService(NodeName string, serviceName string) []int {
var nodeidlist []int
nodeidlist = make([]int, 0)
if NodeName == slf.currentNode.NodeName {
nodeidlist = append(nodeidlist, slf.currentNode.NodeID)
}
v, ok := slf.mapClusterNodeService[NodeName]
if ok == false {
return nodeidlist
}
for _, n := range v {
_, ok = n.ServiceList[serviceName]
if ok == true {
nodeidlist = append(nodeidlist, n.NodeID)
}
}
return nodeidlist
}
func (slf *ClusterConfig) HasLocalService(serviceName string) bool {
_, ok := slf.currentNode.ServiceList[serviceName]
return ok == true
}
func IsExistsNode(nodelist []CNode, pNode *CNode) bool {
for _, node := range nodelist {
if node.NodeID == pNode.NodeID {
return true
}
}
return false
}
func (slf *ClusterConfig) GetNodeNameByNodeId(nodeid int) string {
node, ok := slf.mapIdNode[nodeid]
if ok == false {
return ""
}
return node.NodeName
}
func (slf *ClusterConfig) GetClusterMode() string {
//SubNet []SubNetNodeInfo
for _, subnet := range slf.SubNet {
if subnet.SubNetName == slf.currentNode.SubNetName {
return subnet.SubNetMode
}
}
return ""
}
type CACfg struct {
CertFile string
KeyFile string
}
//NodeConfig ...
type NodeData struct {
NodeID int
LogLevel uint
HttpPort uint16
WSPort uint16
ListenPort uint16
NodeAddr string
CAFile []CACfg
Environment string
IsListenLog int
IsSendErrorMail int
}
type NodeConfig struct {
Public NodeData
NodeList []NodeData
}
//ReadNodeConfig ...
func ReadAllNodeConfig(path string) (map[int]NodeData, error) {
c := &NodeConfig{}
d, err := ioutil.ReadFile(path)
if err != nil {
return nil, err
}
err = json.Unmarshal(d, c)
if err != nil {
return nil, err
}
var mapNodeData map[int]NodeData
mapNodeData = map[int]NodeData{}
//data = c.Public
for _, v := range c.NodeList {
mapNodeData[v.NodeID] = v
}
return mapNodeData, nil
}
func ReadNodeConfig(path string, nodeid int) (*NodeData, error) {
c := &NodeConfig{}
d, err := ioutil.ReadFile(path)
if err != nil {
return nil, err
}
err = json.Unmarshal(d, c)
if err != nil {
return nil, err
}
var data NodeData
data = c.Public
for _, v := range c.NodeList {
if v.NodeID == nodeid {
data = v
if v.Environment == "" && c.Public.Environment != "" {
data.Environment = c.Public.Environment
}
if len(v.CAFile) == 0 && len(c.Public.CAFile) != 0 {
data.CAFile = c.Public.CAFile
}
if v.HttpPort == 0 && c.Public.HttpPort != 0 {
data.HttpPort = c.Public.HttpPort
}
if v.WSPort == 0 && c.Public.WSPort != 0 {
data.WSPort = c.Public.WSPort
}
if v.LogLevel == 0 && c.Public.LogLevel != 0 {
data.LogLevel = c.Public.LogLevel
}
if v.IsListenLog == 0 && c.Public.IsListenLog != 0 {
data.IsListenLog = c.Public.IsListenLog
}
break
}
}
return &data, nil
}

192
cluster/parsecfg.go Normal file
View File

@@ -0,0 +1,192 @@
package cluster
import (
"encoding/json"
"fmt"
"io/ioutil"
"strings"
)
func (slf *Cluster) ReadClusterConfig(filepath string) (*SubNet,error) {
c := &SubNet{}
d, err := ioutil.ReadFile(filepath)
if err != nil {
return nil, err
}
err = json.Unmarshal(d, c)
if err != nil {
return nil, err
}
return c,nil
}
func (slf *Cluster) ReadServiceConfig(filepath string) (map[string]interface{},error) {
c := map[string]interface{}{}
d, err := ioutil.ReadFile(filepath)
if err != nil {
return nil, err
}
err = json.Unmarshal(d, &c)
if err != nil {
return nil, err
}
return c,nil
}
func (slf *Cluster) ReadAllSubNetConfig() error {
clusterCfgPath :=strings.TrimRight(configdir,"/") +"/cluster"
fileInfoList,err := ioutil.ReadDir(clusterCfgPath)
if err != nil {
return err
}
slf.mapSubNetInfo =map[string] SubNet{}
for _,f := range fileInfoList{
if f.IsDir() == true {
subnetinfo,err:=slf.ReadClusterConfig(strings.TrimRight(strings.TrimRight(clusterCfgPath,"/"),"\\")+"/"+f.Name()+"/"+"cluster.json")
if err != nil {
return err
}
slf.mapSubNetInfo[f.Name()] = *subnetinfo
}
}
return nil
}
func (slf *Cluster) ReadLocalSubNetServiceConfig(subnet string) error {
clusterCfgPath :=strings.TrimRight(configdir,"/") +"/cluster"
fileInfoList,err := ioutil.ReadDir(clusterCfgPath)
if err != nil {
return err
}
slf.mapSubNetInfo =map[string] SubNet{}
for _,f := range fileInfoList{
if f.IsDir() == true && f.Name()==subnet{ //同一子网
localNodeServiceCfg,err:=slf.ReadServiceConfig(strings.TrimRight(strings.TrimRight(clusterCfgPath,"/"),"\\")+"/"+f.Name()+"/"+"service.json")
if err != nil {
return err
}
slf.localNodeServiceCfg =localNodeServiceCfg
}
}
return nil
}
func (slf *Cluster) InitCfg(currentNodeId int) error{
//mapSubNetInfo := map[string] SubNet{} //子网名称，子网信息
mapSubNetNodeInfo := map[string]map[int]NodeInfo{} //map[子网名称]map[NodeId]NodeInfo
localSubNetMapNode := map[int]NodeInfo{} //本子网内 map[NodeId]NodeInfo
localSubNetMapService := map[string][]NodeInfo{} //本子网内所有ServiceName对应的结点列表
localNodeMapService := map[string]interface{}{} //本Node支持的服务
localNodeInfo := NodeInfo{}
err := slf.ReadAllSubNetConfig()
//分析配置
var localSubnetName string
for subnetName,subnetInfo := range slf.mapSubNetInfo {
for _,nodeinfo := range subnetInfo.NodeList {
//装载slf.mapNodeInfo
_,ok := mapSubNetNodeInfo[subnetName]
if ok == false {
mapnodeInfo := make(map[int]NodeInfo,1)
mapnodeInfo[nodeinfo.NodeId] = nodeinfo
mapSubNetNodeInfo[subnetName] = mapnodeInfo
}else{
mapSubNetNodeInfo[subnetName][nodeinfo.NodeId] = nodeinfo
}
//判断本进程的子网
if nodeinfo.NodeId == currentNodeId {
localSubnetName = subnetName
}
}
}
//装载
subnet,ok := slf.mapSubNetInfo[localSubnetName]
if ok == false {
return fmt.Errorf("NodeId %d not in any subnet",currentNodeId)
}
for _,nodeinfo := range subnet.NodeList {
localSubNetMapNode[nodeinfo.NodeId] = nodeinfo
//装载本Node进程所有的服务
if nodeinfo.NodeId == currentNodeId {
for _,s := range nodeinfo.ServiceList {
servicename := s
if strings.Index(s,"_") == 0 {
servicename = s[1:]
}
localNodeMapService[servicename] = nil
}
localNodeInfo = nodeinfo
}
for _,s := range nodeinfo.ServiceList {
//以_打头的，表示只在本机进程，不对整个子网开发
if strings.Index(s,"_") == 0 {
continue
}
if _,ok := localSubNetMapService[s];ok== true{
localSubNetMapService[s] = []NodeInfo{}
}
localSubNetMapService[s] = append(localSubNetMapService[s],nodeinfo)
}
}
if localNodeInfo.NodeId == 0 {
return fmt.Errorf("Canoot find NodeId %d not in any config file.",currentNodeId)
}
slf.mapSubNetNodeInfo=mapSubNetNodeInfo
slf.localSubNetMapNode=localSubNetMapNode
slf.localSubNetMapService = localSubNetMapService
slf.localNodeMapService = localNodeMapService
slf.localsubnet = subnet
slf.localNodeInfo =localNodeInfo
//读取服务
return err
}
func (slf *Cluster) IsConfigService(servicename string) bool {
_,ok := slf.localNodeMapService[servicename]
return ok
}
func (slf *Cluster) GetNodeIdByService(servicename string) []int{
var nodelist []int
nodeInfoList,ok := slf.localSubNetMapService[servicename]
if ok == true {
for _,node := range nodeInfoList {
nodelist = append(nodelist,node.NodeId)
}
}
return nodelist
}
func (slf *Cluster) GetServiceCfg(servicename string) interface{}{
v,ok := slf.localNodeServiceCfg[servicename]
if ok == false{
return nil
}
return v
}

BIN
doc/origin分布式框架.pptx
View File

Binary file not shown.

80
event/event.go Normal file
View File

@@ -0,0 +1,80 @@
package event
import (
"github.com/duanhf2012/originnet/log"
"sync"
)
const Default_EventChannelLen = 10000
//事件接受器
type EventReciver func(event *Event) error
type Event struct {
Type EventType
Data interface{}
}
type IEventProcessor interface {
NotifyEvent(*Event)
OnEventHandler(event *Event) error
SetEventReciver(eventProcessor IEventProcessor)
GetEventReciver() IEventProcessor
SetEventChanNum(num int32) bool
}
type EventProcessor struct {
//事件管道
EventChan chan *Event
eventReciver IEventProcessor
eventChanNumLocker sync.RWMutex
eventChanNum int32
}
func (slf *EventProcessor) NotifyEvent(pEvent *Event) {
if len(slf.EventChan) >= int(slf.eventChanNum) {
log.Error("event queue is full!")
}
slf.EventChan <-pEvent
}
func (slf *EventProcessor) OnEventHandler(event *Event) error{
return nil
}
func (slf *EventProcessor) GetEventChan() chan *Event{
slf.eventChanNumLocker.Lock()
defer slf.eventChanNumLocker.Unlock()
if slf.eventChanNum == 0 {
slf.eventChanNum = Default_EventChannelLen
}
if slf.EventChan == nil {
slf.EventChan = make(chan *Event,slf.eventChanNum)
}
return slf.EventChan
}
//不允许重复设置
func (slf *EventProcessor) SetEventChanNum(num int32) bool {
slf.eventChanNumLocker.Lock()
defer slf.eventChanNumLocker.Unlock()
if slf.eventChanNum>0 {
return false
}
slf.eventChanNum = num
return true
}
func (slf *EventProcessor) SetEventReciver(eventProcessor IEventProcessor){
slf.eventReciver = eventProcessor
}
func (slf *EventProcessor) GetEventReciver() IEventProcessor{
return slf.eventReciver
}

14
event/eventtype.go Normal file
View File

@@ -0,0 +1,14 @@
package event
type EventType int
//大于Sys_Event_User_Define给用户定义
const (
Sys_Event_Tcp_Connected EventType= 1
Sys_Event_Tcp_DisConnected EventType= 2
Sys_Event_Tcp_RecvPack EventType = 3
Sys_Event_User_Define EventType = 1000
)

43
example/GateService/GateService.go Normal file
View File

@@ -0,0 +1,43 @@
package GateService
import (
"github.com/duanhf2012/originnet/event"
"github.com/duanhf2012/originnet/node"
"github.com/duanhf2012/originnet/service"
"github.com/duanhf2012/originnet/sysservice"
)
type GateService struct {
processor *PBProcessor
service.Service
}
func (slf *GateService) OnInit() error{
tcpervice := node.GetService("TcpService").(*sysservice.TcpService)
tcpervice.SetProcessor(&PBProcessor{})
return nil
}
func (slf *GateService) OnEventHandler(ev *event.Event) error{
if ev.Type == event.Sys_Event_Tcp_RecvPack {
pPack := ev.Data.(*sysservice.TcpPack)
slf.processor.Route(pPack.Data,pPack.ClientId)
}else if ev.Type == event.Sys_Event_Tcp_Connected {
pPack := ev.Data.(*sysservice.TcpPack)
slf.OnConnected(pPack.ClientId)
}else if ev.Type == event.Sys_Event_Tcp_DisConnected {
pPack := ev.Data.(*sysservice.TcpPack)
slf.OnDisconnected(pPack.ClientId)
}
return nil
}
func (slf *GateService) OnConnected(clientid uint64){
}
func (slf *GateService) OnDisconnected(clientid uint64){
}

79
example/GateService/pbprocessor.go Normal file
View File

@@ -0,0 +1,79 @@
package GateService
import (
"encoding/binary"
"fmt"
"github.com/golang/protobuf/proto"
"reflect"
)
type MessageInfo struct {
msgType reflect.Type
msgHandler MessageHandler
}
type PBProcessor struct {
mapMsg map[uint16]MessageInfo
LittleEndian bool
}
func (slf *PBProcessor) SetLittleEndian(littleEndian bool){
slf.LittleEndian = littleEndian
}
type PackInfo struct {
typ uint16
msg proto.Message
}
// must goroutine safe
func (slf *PBProcessor ) Route(msg interface{},userdata interface{}) error{
pPackInfo := msg.(*PackInfo)
v,ok := slf.mapMsg[pPackInfo.typ]
if ok == false {
return fmt.Errorf("cannot find msgtype %d is register!",pPackInfo.typ)
}
v.msgHandler(userdata.(uint64),pPackInfo.msg)
return nil
}
// must goroutine safe
func (slf *PBProcessor ) Unmarshal(data []byte) (interface{}, error) {
var msgType uint16
if slf.LittleEndian == true {
msgType = binary.LittleEndian.Uint16(data[:2])
}else{
msgType = binary.BigEndian.Uint16(data[:2])
}
info,ok := slf.mapMsg[msgType]
if ok == false {
return nil,fmt.Errorf("cannot find register %d msgtype!",msgType)
}
msg := reflect.New(info.msgType.Elem()).Interface()
protoMsg := msg.(proto.Message)
err := proto.Unmarshal(data[2:], protoMsg)
if err != nil {
return nil,err
}
return &PackInfo{typ:msgType,msg:protoMsg},nil
}
// must goroutine safe
func (slf *PBProcessor ) Marshal(msg interface{}) ([]byte, error){
return proto.Marshal(msg.(proto.Message))
}
type MessageHandler func(clientid uint64,msg proto.Message)
func (slf *PBProcessor) Register(msgtype uint16,msg proto.Message,handle MessageHandler) {
var info MessageInfo
info.msgType = reflect.TypeOf(msg.(proto.Message))
info.msgHandler = handle
slf.mapMsg[msgtype] = info
}

23
example/config/cluster/subnet/cluster.json Normal file
View File

@@ -0,0 +1,23 @@
{
"NodeList":[
{
"NodeId": 1,
"ListenAddr":"127.0.0.1:8001",
"NodeName": "Node_Test1",
"remark":"//以_打头的，表示只在本机进程，不对整个子网开发",
"ServiceList": ["TestService1","TestService2","TestServiceCall"]
},
{
"NodeId": 2,
"ListenAddr":"127.0.0.1:8002",
"NodeName": "Node_Gate2",
"ServiceList": ["SubNet1_Service"]
},
{
"NodeId": 3,
"ListenAddr":"127.0.0.1:8003",
"NodeName": "Node_Room",
"ServiceList": ["SubNet1_Service"]
}
]
}

13
example/config/cluster/subnet/service.json Normal file
View File

@@ -0,0 +1,13 @@
{
"HttpService":{
"Port":10001
},
"TcpService":{
"ListenAddr":"0.0.0.0:9030",
"MaxConnNum":3000,
"PendingWriteNum":10000,
"LittleEndian":false,
"MinMsgLen":4,
"MaxMsgLen":65535
}
}

1
example/main.exe.pid Normal file
View File

@@ -0,0 +1 @@
49288

202
example/main.go Normal file
View File

@@ -0,0 +1,202 @@
package main
import (
"fmt"
"github.com/duanhf2012/originnet/example/GateService"
"github.com/duanhf2012/originnet/node"
"github.com/duanhf2012/originnet/service"
"github.com/duanhf2012/originnet/sysmodule"
"github.com/duanhf2012/originnet/sysservice"
"time"
)
type TestService1 struct {
service.Service
}
type TestService2 struct {
service.Service
}
type TestServiceCall struct {
service.Service
dbModule sysmodule.DBModule
}
func init(){
node.Setup(&TestService1{},&TestService2{},&TestServiceCall{})
}
type Module1 struct{
service.Module
}
type Module2 struct{
service.Module
}
type Module3 struct{
service.Module
}
type Module4 struct{
service.Module
}
var moduleid1 int64
var moduleid2 int64
var moduleid3 int64
var moduleid4 int64
func (slf *Module1) OnInit() error {
fmt.Printf("I'm Module1:%d\n",slf.GetModuleId())
return nil
}
func (slf *Module2) OnInit() error {
fmt.Printf("I'm Module2:%d\n",slf.GetModuleId())
moduleid3,_ = slf.AddModule(&Module3{})
return nil
}
func (slf *Module3) OnInit() error {
fmt.Printf("I'm Module3:%d\n",slf.GetModuleId())
moduleid4,_ = slf.AddModule(&Module4{})
return nil
}
func (slf *Module4) OnInit() error {
fmt.Printf("I'm Module4:%d\n",slf.GetModuleId())
//pService := slf.GetService().(*TestServiceCall)
//pService.RPC_Test(nil,nil)
slf.AfterFunc(time.Second*10,slf.TimerTest)
return nil
}
func (slf *Module4) TimerTest(){
fmt.Printf("Module4 tigger timer\n")
}
func (slf *Module1) OnRelease() {
fmt.Printf("Release Module1:%d\n",slf.GetModuleId())
}
func (slf *Module2) OnRelease() {
fmt.Printf("Release Module2:%d\n",slf.GetModuleId())
}
func (slf *Module3) OnRelease() {
fmt.Printf("Release Module3:%d\n",slf.GetModuleId())
}
func (slf *Module4) OnRelease() {
fmt.Printf("Release Module4:%d\n",slf.GetModuleId())
}
func (slf *TestServiceCall) OnInit() error {
slf.AfterFunc(time.Second*1,slf.Run)
moduleid1,_ = slf.AddModule(&Module1{})
moduleid2,_ = slf.AddModule(&Module2{})
fmt.Print(moduleid1,moduleid2)
slf.dbModule = sysmodule.DBModule{}
slf.dbModule.Init(10,3, "192.168.0.5:3306", "root", "Root!!2018", "Matrix")
slf.dbModule.SetQuerySlowTime(time.Second * 3)
slf.AddModule(&slf.dbModule)
slf.AfterFunc(time.Second*5,slf.Release)
slf.AfterFunc(time.Second, slf.TestDB)
return nil
}
func (slf *TestServiceCall) Release(){
slf.ReleaseModule(moduleid1)
slf.ReleaseModule(moduleid2)
}
type Param struct {
Index int
A int
B string
Pa []string
}
func (slf *TestServiceCall) Run(){
//var ret int
var input int = 10000
bT := time.Now() // 开始时间
//err := slf.Call("TestServiceCall.RPC_Test",&ret,&input)
var param Param
param.A = 2342342341
param.B = "xxxxxxxxxxxxxxxxxxxxxxx"
param.Pa = []string{"ccccc","asfsdfsdaf","bbadfsdf","ewrwefasdf","safsadfka;fksd"}
for i:=input;i>=0;i--{
param.Index = i
slf.AsyncCall("TestService1.RPC_Test",&param, func(reply *Param, err error) {
if reply.Index == 0 || err != nil{
eT := time.Since(bT) // 从开始到当前所消耗的时间
fmt.Print(err,eT.Milliseconds())
fmt.Print("..................",eT,"\n")
}
//fmt.Print(*reply,"\n",err)
})
}
fmt.Print("finsh....")
}
func (slf *TestService1) RPC_Test(a *Param,b *Param) error {
*a = *b
return nil
}
func (slf *TestService1) OnInit() error {
return nil
}
func (slf *TestServiceCall) RPC_Test(a *int,b *int) error {
fmt.Printf("TestService2\n")
*a = *b
return nil
}
func (slf *TestServiceCall) TestDB() {
assetsInfo := &struct {
Cash int64 `json:"cash"` //美金余额 100
Gold int64 `json:"gold"` //金币余额
Heart int64 `json:"heart"` //心数
}{}
sql := `call sp_select_userAssets(?)`
userID := 100000802
err := slf.dbModule.AsyncQuery(func(dataList *sysmodule.DataSetList, err error) {
if err != nil {
return
}
err = dataList.UnMarshal(assetsInfo)
if err != nil {
return
}
},-1, sql, &userID)
fmt.Println(err)
}
func (slf *TestService2) OnInit() error {
return nil
}
func main(){
node.Init()
tcpService := &sysservice.TcpService{}
gateService := &GateService.GateService{}
tcpService.SetEventReciver(gateService)
node.Setup(tcpService,gateService)
node.Start()
}

64
example/serviceTest/OriginServeOne.go Normal file
View File

@@ -0,0 +1,64 @@
package serviceTest
import (
"fmt"
"github.com/duanhf2012/originnet/service"
"time"
)
type TestAsyn struct {
a int
b string
}
type OriginServerOne struct {
service.Service
}
func (slf *OriginServerOne) OnInit() error {
//slf.AfterFunc(time.Second,slf.testCall)
//slf.AfterFunc(time.Second*5,slf.testCall)
//slf.AfterFunc(time.Second*10, slf.testGRCall)
//slf.AfterFunc(time.Second*15, slf.testGRCall)
//slf.AfterFunc(time.Second, slf.testAsyncCall)
slf.AfterFunc(time.Second, slf.testAsyncGRCall)
return nil
}
func (slf *OriginServerOne) testCall() {
a := 1
b := 10
slf.Call("OriginServerTwo.RPC_TestCall", &a, &b)
fmt.Println(b)
}
func (slf *OriginServerOne) testGRCall() {
a := 1
b := 10
slf.GRCall("OriginServerTwo.RPC_TestCall", &a, &b)
fmt.Println(b)
}
func (slf *OriginServerOne) testAsyncCall() {
for i := 0; i < 100; i++ {
in := i
bT := time.Now()
slf.AsyncCall("OriginServerTwo.RPC_TestAsyncCall", &in, func(reply *TestAsyn, err error) {
eT := time.Since(bT) // 从开始到当前所消耗的时间
fmt.Println(reply, eT)
})
fmt.Println(in)
}
}
func (slf *OriginServerOne) testAsyncGRCall() {
for i := 0; i < 100; i++ {
in := i
bT := time.Now()
slf.GRAsyncCall("OriginServerTwo.RPC_TestAsyncCall", &in, func(reply *TestAsyn, err error) {
eT := time.Since(bT)
fmt.Println(reply, eT)
})
fmt.Println(in)
}
}

27
example/serviceTest/OriginServerTwo.go Normal file
View File

@@ -0,0 +1,27 @@
package serviceTest
import (
"fmt"
"github.com/duanhf2012/originnet/service"
"time"
)
type OriginServerTwo struct {
service.Service
}
func (slf *OriginServerTwo) RPC_TestCall(a *int,b *int) error {
fmt.Printf("OriginServerTwo\n")
*a = *b*2
//slf.AfterFunc(time.Second,slf.Test)
return nil
}
func (slf *OriginServerTwo) RPC_TestAsyncCall(a *int, reply *TestAsyn) error {
fmt.Printf("OriginServerTwo async start sleep %d\n", *a)
time.Sleep(time.Second)
reply.a = *a
reply.b = "fuck!"
return nil
}

29
log/example_test.go Normal file
View File

@@ -0,0 +1,29 @@
package log_test
import (
"github.com/name5566/leaf/log"
l "log"
)
func Example() {
name := "Leaf"
log.Debug("My name is %v", name)
log.Release("My name is %v", name)
log.Error("My name is %v", name)
// log.Fatal("My name is %v", name)
logger, err := log.New("release", "", l.LstdFlags)
if err != nil {
return
}
defer logger.Close()
logger.Debug("will not print")
logger.Release("My name is %v", name)
log.Export(logger)
log.Debug("will not print")
log.Release("My name is %v", name)
}

153
log/log.go Normal file
View File

@@ -0,0 +1,153 @@
package log
import (
"errors"
"fmt"
"log"
"os"
"path"
"strings"
"time"
)
// levels
const (
debugLevel = 0
releaseLevel = 1
errorLevel = 2
fatalLevel = 3
)
const (
printDebugLevel = "[debug ] "
printReleaseLevel = "[release] "
printErrorLevel = "[error ] "
printFatalLevel = "[fatal ] "
)
type Logger struct {
level int
baseLogger *log.Logger
baseFile *os.File
}
func New(strLevel string, pathname string, flag int) (*Logger, error) {
// level
var level int
switch strings.ToLower(strLevel) {
case "debug":
level = debugLevel
case "release":
level = releaseLevel
case "error":
level = errorLevel
case "fatal":
level = fatalLevel
default:
return nil, errors.New("unknown level: " + strLevel)
}
// logger
var baseLogger *log.Logger
var baseFile *os.File
if pathname != "" {
now := time.Now()
filename := fmt.Sprintf("%d%02d%02d_%02d_%02d_%02d.log",
now.Year(),
now.Month(),
now.Day(),
now.Hour(),
now.Minute(),
now.Second())
file, err := os.Create(path.Join(pathname, filename))
if err != nil {
return nil, err
}
baseLogger = log.New(file, "", flag)
baseFile = file
} else {
baseLogger = log.New(os.Stdout, "", flag)
}
// new
logger := new(Logger)
logger.level = level
logger.baseLogger = baseLogger
logger.baseFile = baseFile
return logger, nil
}
// It's dangerous to call the method on logging
func (logger *Logger) Close() {
if logger.baseFile != nil {
logger.baseFile.Close()
}
logger.baseLogger = nil
logger.baseFile = nil
}
func (logger *Logger) doPrintf(level int, printLevel string, format string, a ...interface{}) {
if level < logger.level {
return
}
if logger.baseLogger == nil {
panic("logger closed")
}
format = printLevel + format
logger.baseLogger.Output(3, fmt.Sprintf(format, a...))
if level == fatalLevel {
os.Exit(1)
}
}
func (logger *Logger) Debug(format string, a ...interface{}) {
logger.doPrintf(debugLevel, printDebugLevel, format, a...)
}
func (logger *Logger) Release(format string, a ...interface{}) {
logger.doPrintf(releaseLevel, printReleaseLevel, format, a...)
}
func (logger *Logger) Error(format string, a ...interface{}) {
logger.doPrintf(errorLevel, printErrorLevel, format, a...)
}
func (logger *Logger) Fatal(format string, a ...interface{}) {
logger.doPrintf(fatalLevel, printFatalLevel, format, a...)
}
var gLogger, _ = New("debug", "", log.LstdFlags)
// It's dangerous to call the method on logging
func Export(logger *Logger) {
if logger != nil {
gLogger = logger
}
}
func Debug(format string, a ...interface{}) {
gLogger.doPrintf(debugLevel, printDebugLevel, format, a...)
}
func Release(format string, a ...interface{}) {
gLogger.doPrintf(releaseLevel, printReleaseLevel, format, a...)
}
func Error(format string, a ...interface{}) {
gLogger.doPrintf(errorLevel, printErrorLevel, format, a...)
}
func Fatal(format string, a ...interface{}) {
gLogger.doPrintf(fatalLevel, printFatalLevel, format, a...)
}
func Close() {
gLogger.Close()
}

6
network/agent.go Normal file
View File

@@ -0,0 +1,6 @@
package network
type Agent interface {
Run()
OnClose()
}

14
network/conn.go Normal file
View File

@@ -0,0 +1,14 @@
package network
import (
"net"
)
type Conn interface {
ReadMsg() ([]byte, error)
WriteMsg(args ...[]byte) error
LocalAddr() net.Addr
RemoteAddr() net.Addr
Close()
Destroy()
}

98
network/httpserver.go
View File

@@ -1,98 +0,0 @@
package network
import (
"crypto/tls"
"fmt"
"net/http"
"os"
"time"
"github.com/duanhf2012/origin/service"
"github.com/duanhf2012/origin/sysmodule"
)
type CA struct {
certfile string
keyfile string
}
type HttpServer struct {
port uint16
handler http.Handler
readtimeout time.Duration
writetimeout time.Duration
httpserver *http.Server
caList []CA
ishttps bool
}
func (slf *HttpServer) Init(port uint16, handler http.Handler, readtimeout time.Duration, writetimeout time.Duration) {
slf.port = port
slf.handler = handler
slf.readtimeout = readtimeout
slf.writetimeout = writetimeout
}
func (slf *HttpServer) HandleFunc(pattern string, handler func(http.ResponseWriter, *http.Request)) {
http.HandleFunc(pattern, handler)
}
func (slf *HttpServer) Start() {
go slf.startListen()
}
func (slf *HttpServer) startListen() error {
listenPort := fmt.Sprintf(":%d", slf.port)
var tlscatList []tls.Certificate
var tlsConfig *tls.Config
for _, cadata := range slf.caList {
cer, err := tls.LoadX509KeyPair(cadata.certfile, cadata.keyfile)
if err != nil {
service.GetLogger().Printf(sysmodule.LEVER_FATAL, "load CA [%s]-[%s] file is error :%s", cadata.certfile, cadata.keyfile, err.Error())
os.Exit(1)
return nil
}
tlscatList = append(tlscatList, cer)
}
if len(tlscatList) > 0 {
tlsConfig = &tls.Config{Certificates: tlscatList}
}
slf.httpserver = &http.Server{
Addr: listenPort,
Handler: slf.handler,
ReadTimeout: 10 * time.Second,
WriteTimeout: 10 * time.Second,
MaxHeaderBytes: 1 << 20,
TLSConfig: tlsConfig,
}
var err error
if slf.ishttps == true {
err = slf.httpserver.ListenAndServeTLS("", "")
} else {
err = slf.httpserver.ListenAndServe()
}
if err != nil {
service.GetLogger().Printf(sysmodule.LEVER_FATAL, "http.ListenAndServe(%d, nil) error:%v\n", listenPort, err)
fmt.Printf("http.ListenAndServe(%d, %v) error\n", slf.port, err)
os.Exit(1)
}
return nil
}
func (slf *HttpServer) SetHttps(certfile string, keyfile string) bool {
if certfile == "" || keyfile == "" {
return false
}
slf.caList = append(slf.caList, CA{certfile, keyfile})
slf.ishttps = true
return true
}

7
network/netserver.go Normal file
View File

@@ -0,0 +1,7 @@
package network
type inetserver interface {
}

10
network/processor.go Normal file
View File

@@ -0,0 +1,10 @@
package network
type Processor interface {
// must goroutine safe
Route(msg interface{}, userData interface{}) error
// must goroutine safe
Unmarshal(data []byte) (interface{}, error)
// must goroutine safe
Marshal(msg interface{}) ([]byte, error)
}

1
network/processor/gobprocessor.go Normal file
View File

@@ -0,0 +1 @@
package processor

17
network/processor/jsonprocessor.go Normal file
View File

@@ -0,0 +1,17 @@
package processor
type JsonProcessor struct {
//SetByteOrder(littleEndian bool)
//SetMsgLen(lenMsgLen int, minMsgLen uint32, maxMsgLen uint32)
}
func (slf *JsonProcessor) Unmarshal(data []byte) (interface{}, error) {
return nil,nil
}
func (slf *JsonProcessor) Marshal(msg interface{}) ([][]byte, error) {
return nil,nil
}

1
network/processor/msgpackprocessor.go Normal file
View File

@@ -0,0 +1 @@
package processor

11
network/processor/processor.go Normal file
View File

@@ -0,0 +1,11 @@
package processor
type IProcessor interface {
//SetByteOrder(littleEndian bool)
//SetMsgLen(lenMsgLen int, minMsgLen uint32, maxMsgLen uint32)
Unmarshal(data []byte) (interface{}, error)
// must goroutine safe
Marshal(msg interface{}) ([][]byte, error)
}

1
network/processor/protobufprocessor.go Normal file
View File

@@ -0,0 +1 @@
package processor

130
network/tcp_client.go Normal file
View File

@@ -0,0 +1,130 @@
package network
import (
"github.com/duanhf2012/originnet/log"
"net"
"sync"
"time"
)
type TCPClient struct {
sync.Mutex
Addr string
ConnNum int
ConnectInterval time.Duration
PendingWriteNum int
AutoReconnect bool
NewAgent func(*TCPConn) Agent
conns ConnSet
wg sync.WaitGroup
closeFlag bool
// msg parser
LenMsgLen int
MinMsgLen uint32
MaxMsgLen uint32
LittleEndian bool
msgParser *MsgParser
}
func (client *TCPClient) Start() {
client.init()
for i := 0; i < client.ConnNum; i++ {
client.wg.Add(1)
go client.connect()
}
}
func (client *TCPClient) init() {
client.Lock()
defer client.Unlock()
if client.ConnNum <= 0 {
client.ConnNum = 1
log.Release("invalid ConnNum, reset to %v", client.ConnNum)
}
if client.ConnectInterval <= 0 {
client.ConnectInterval = 3 * time.Second
log.Release("invalid ConnectInterval, reset to %v", client.ConnectInterval)
}
if client.PendingWriteNum <= 0 {
client.PendingWriteNum = 100
log.Release("invalid PendingWriteNum, reset to %v", client.PendingWriteNum)
}
if client.NewAgent == nil {
log.Fatal("NewAgent must not be nil")
}
if client.conns != nil {
log.Fatal("client is running")
}
client.conns = make(ConnSet)
client.closeFlag = false
// msg parser
msgParser := NewMsgParser()
msgParser.SetMsgLen(client.LenMsgLen, client.MinMsgLen, client.MaxMsgLen)
msgParser.SetByteOrder(client.LittleEndian)
client.msgParser = msgParser
}
func (client *TCPClient) dial() net.Conn {
for {
conn, err := net.Dial("tcp", client.Addr)
if err == nil || client.closeFlag {
return conn
}
log.Release("connect to %v error: %v", client.Addr, err)
time.Sleep(client.ConnectInterval)
continue
}
}
func (client *TCPClient) connect() {
defer client.wg.Done()
reconnect:
conn := client.dial()
if conn == nil {
return
}
client.Lock()
if client.closeFlag {
client.Unlock()
conn.Close()
return
}
client.conns[conn] = struct{}{}
client.Unlock()
tcpConn := newTCPConn(conn, client.PendingWriteNum, client.msgParser)
agent := client.NewAgent(tcpConn)
agent.Run()
// cleanup
tcpConn.Close()
client.Lock()
delete(client.conns, conn)
client.Unlock()
agent.OnClose()
if client.AutoReconnect {
time.Sleep(client.ConnectInterval)
goto reconnect
}
}
func (client *TCPClient) Close() {
client.Lock()
client.closeFlag = true
for conn := range client.conns {
conn.Close()
}
client.conns = nil
client.Unlock()
client.wg.Wait()
}

113
network/tcp_conn.go Normal file
View File

@@ -0,0 +1,113 @@
package network
import (
"github.com/duanhf2012/originnet/log"
"net"
"sync"
)
type ConnSet map[net.Conn]struct{}
type TCPConn struct {
sync.Mutex
conn net.Conn
writeChan chan []byte
closeFlag bool
msgParser *MsgParser
}
func newTCPConn(conn net.Conn, pendingWriteNum int, msgParser *MsgParser) *TCPConn {
tcpConn := new(TCPConn)
tcpConn.conn = conn
tcpConn.writeChan = make(chan []byte, pendingWriteNum)
tcpConn.msgParser = msgParser
go func() {
for b := range tcpConn.writeChan {
if b == nil {
break
}
_, err := conn.Write(b)
if err != nil {
break
}
}
conn.Close()
tcpConn.Lock()
tcpConn.closeFlag = true
tcpConn.Unlock()
}()
return tcpConn
}
func (tcpConn *TCPConn) doDestroy() {
tcpConn.conn.(*net.TCPConn).SetLinger(0)
tcpConn.conn.Close()
if !tcpConn.closeFlag {
close(tcpConn.writeChan)
tcpConn.closeFlag = true
}
}
func (tcpConn *TCPConn) Destroy() {
tcpConn.Lock()
defer tcpConn.Unlock()
tcpConn.doDestroy()
}
func (tcpConn *TCPConn) Close() {
tcpConn.Lock()
defer tcpConn.Unlock()
if tcpConn.closeFlag {
return
}
tcpConn.doWrite(nil)
tcpConn.closeFlag = true
}
func (tcpConn *TCPConn) doWrite(b []byte) {
if len(tcpConn.writeChan) == cap(tcpConn.writeChan) {
log.Debug("close conn: channel full")
tcpConn.doDestroy()
return
}
tcpConn.writeChan <- b
}
// b must not be modified by the others goroutines
func (tcpConn *TCPConn) Write(b []byte) {
tcpConn.Lock()
defer tcpConn.Unlock()
if tcpConn.closeFlag || b == nil {
return
}
tcpConn.doWrite(b)
}
func (tcpConn *TCPConn) Read(b []byte) (int, error) {
return tcpConn.conn.Read(b)
}
func (tcpConn *TCPConn) LocalAddr() net.Addr {
return tcpConn.conn.LocalAddr()
}
func (tcpConn *TCPConn) RemoteAddr() net.Addr {
return tcpConn.conn.RemoteAddr()
}
func (tcpConn *TCPConn) ReadMsg() ([]byte, error) {
return tcpConn.msgParser.Read(tcpConn)
}
func (tcpConn *TCPConn) WriteMsg(args ...[]byte) error {
return tcpConn.msgParser.Write(tcpConn, args...)
}

154
network/tcp_msg.go Normal file
View File

@@ -0,0 +1,154 @@
package network
import (
"encoding/binary"
"errors"
"io"
"math"
)
// --------------
// | len | data |
// --------------
type MsgParser struct {
lenMsgLen int
minMsgLen uint32
maxMsgLen uint32
littleEndian bool
}
func NewMsgParser() *MsgParser {
p := new(MsgParser)
p.lenMsgLen = 2
p.minMsgLen = 1
p.maxMsgLen = 4096
p.littleEndian = false
return p
}
// It's dangerous to call the method on reading or writing
func (p *MsgParser) SetMsgLen(lenMsgLen int, minMsgLen uint32, maxMsgLen uint32) {
if lenMsgLen == 1 || lenMsgLen == 2 || lenMsgLen == 4 {
p.lenMsgLen = lenMsgLen
}
if minMsgLen != 0 {
p.minMsgLen = minMsgLen
}
if maxMsgLen != 0 {
p.maxMsgLen = maxMsgLen
}
var max uint32
switch p.lenMsgLen {
case 1:
max = math.MaxUint8
case 2:
max = math.MaxUint16
case 4:
max = math.MaxUint32
}
if p.minMsgLen > max {
p.minMsgLen = max
}
if p.maxMsgLen > max {
p.maxMsgLen = max
}
}
// It's dangerous to call the method on reading or writing
func (p *MsgParser) SetByteOrder(littleEndian bool) {
p.littleEndian = littleEndian
}
// goroutine safe
func (p *MsgParser) Read(conn *TCPConn) ([]byte, error) {
var b [4]byte
bufMsgLen := b[:p.lenMsgLen]
// read len
if _, err := io.ReadFull(conn, bufMsgLen); err != nil {
return nil, err
}
// parse len
var msgLen uint32
switch p.lenMsgLen {
case 1:
msgLen = uint32(bufMsgLen[0])
case 2:
if p.littleEndian {
msgLen = uint32(binary.LittleEndian.Uint16(bufMsgLen))
} else {
msgLen = uint32(binary.BigEndian.Uint16(bufMsgLen))
}
case 4:
if p.littleEndian {
msgLen = binary.LittleEndian.Uint32(bufMsgLen)
} else {
msgLen = binary.BigEndian.Uint32(bufMsgLen)
}
}
// check len
if msgLen > p.maxMsgLen {
return nil, errors.New("message too long")
} else if msgLen < p.minMsgLen {
return nil, errors.New("message too short")
}
// data
msgData := make([]byte, msgLen)
if _, err := io.ReadFull(conn, msgData); err != nil {
return nil, err
}
return msgData, nil
}
// goroutine safe
func (p *MsgParser) Write(conn *TCPConn, args ...[]byte) error {
// get len
var msgLen uint32
for i := 0; i < len(args); i++ {
msgLen += uint32(len(args[i]))
}
// check len
if msgLen > p.maxMsgLen {
return errors.New("message too long")
} else if msgLen < p.minMsgLen {
return errors.New("message too short")
}
msg := make([]byte, uint32(p.lenMsgLen)+msgLen)
// write len
switch p.lenMsgLen {
case 1:
msg[0] = byte(msgLen)
case 2:
if p.littleEndian {
binary.LittleEndian.PutUint16(msg, uint16(msgLen))
} else {
binary.BigEndian.PutUint16(msg, uint16(msgLen))
}
case 4:
if p.littleEndian {
binary.LittleEndian.PutUint32(msg, msgLen)
} else {
binary.BigEndian.PutUint32(msg, msgLen)
}
}
// write data
l := p.lenMsgLen
for i := 0; i < len(args); i++ {
copy(msg[l:], args[i])
l += len(args[i])
}
conn.Write(msg)
return nil
}

127
network/tcp_server.go Normal file
View File

@@ -0,0 +1,127 @@
package network
import (
"github.com/duanhf2012/originnet/log"
"net"
"sync"
"time"
)
type TCPServer struct {
Addr string
MaxConnNum int
PendingWriteNum int
NewAgent func(*TCPConn) Agent
ln net.Listener
conns ConnSet
mutexConns sync.Mutex
wgLn sync.WaitGroup
wgConns sync.WaitGroup
// msg parser
LenMsgLen int
MinMsgLen uint32
MaxMsgLen uint32
LittleEndian bool
msgParser *MsgParser
}
func (server *TCPServer) Start() {
server.init()
go server.run()
}
func (server *TCPServer) init() {
ln, err := net.Listen("tcp", server.Addr)
if err != nil {
log.Fatal("%v", err)
}
if server.MaxConnNum <= 0 {
server.MaxConnNum = 100
log.Release("invalid MaxConnNum, reset to %v", server.MaxConnNum)
}
if server.PendingWriteNum <= 0 {
server.PendingWriteNum = 100
log.Release("invalid PendingWriteNum, reset to %v", server.PendingWriteNum)
}
if server.NewAgent == nil {
log.Fatal("NewAgent must not be nil")
}
server.ln = ln
server.conns = make(ConnSet)
// msg parser
msgParser := NewMsgParser()
msgParser.SetMsgLen(server.LenMsgLen, server.MinMsgLen, server.MaxMsgLen)
msgParser.SetByteOrder(server.LittleEndian)
server.msgParser = msgParser
}
func (server *TCPServer) run() {
server.wgLn.Add(1)
defer server.wgLn.Done()
var tempDelay time.Duration
for {
conn, err := server.ln.Accept()
if err != nil {
if ne, ok := err.(net.Error); ok && ne.Temporary() {
if tempDelay == 0 {
tempDelay = 5 * time.Millisecond
} else {
tempDelay *= 2
}
if max := 1 * time.Second; tempDelay > max {
tempDelay = max
}
log.Release("accept error: %v; retrying in %v", err, tempDelay)
time.Sleep(tempDelay)
continue
}
return
}
tempDelay = 0
server.mutexConns.Lock()
if len(server.conns) >= server.MaxConnNum {
server.mutexConns.Unlock()
conn.Close()
log.Debug("too many connections")
continue
}
server.conns[conn] = struct{}{}
server.mutexConns.Unlock()
server.wgConns.Add(1)
tcpConn := newTCPConn(conn, server.PendingWriteNum, server.msgParser)
agent := server.NewAgent(tcpConn)
go func() {
agent.Run()
// cleanup
tcpConn.Close()
server.mutexConns.Lock()
delete(server.conns, conn)
server.mutexConns.Unlock()
agent.OnClose()
server.wgConns.Done()
}()
}
}
func (server *TCPServer) Close() {
server.ln.Close()
server.wgLn.Wait()
server.mutexConns.Lock()
for conn := range server.conns {
conn.Close()
}
server.conns = nil
server.mutexConns.Unlock()
server.wgConns.Wait()
}

48
network/tcpsocketclient.go
View File

@@ -1,48 +0,0 @@
package network
import (
"fmt"
"github.com/golang/protobuf/proto"
"net"
)
type TcpSocketClient struct {
conn net.Conn
addr string
}
func (slf *TcpSocketClient) Connect(addr string) error{
tcpAddr,terr := net.ResolveTCPAddr("tcp",addr)
if terr != nil {
return terr
}
conn,err := net.DialTCP("tcp",nil,tcpAddr)
if err!=nil {
fmt.Println("Client connect error ! " + err.Error())
return err
}
slf.conn = conn
slf.addr = addr
//
return nil
}
func (slf *TcpSocketClient) SendMsg(packtype uint16,message proto.Message) error{
if slf.conn == nil {
return fmt.Errorf("cannt connect %s",slf.addr)
}
var msg MsgBasePack
data,err := proto.Marshal(message)
if err != nil {
return err
}
msg.Make(packtype,data)
slf.conn.Write(msg.Bytes())
return nil
}

331
network/tcpsocketserver.go
View File

@@ -1,331 +0,0 @@
package network
import (
"bufio"
"encoding/binary"
"fmt"
"github.com/duanhf2012/origin/service"
"github.com/duanhf2012/origin/util"
"github.com/golang/protobuf/proto"
"io"
"net"
"unsafe"
"os"
"time"
)
type ITcpSocketServerReciver interface {
OnConnected(pClient *SClient)
OnDisconnect(pClient *SClient)
OnRecvMsg(pClient *SClient, pPack *MsgBasePack)
}
type SClient struct {
id uint64
conn net.Conn
recvPack *util.SyncQueue
sendPack *util.SyncQueue
tcpserver *TcpSocketServer
remoteip string
starttime int64
bClose bool
}
type TcpSocketServer struct {
listenAddr string //ip:port
mapClient util.Map
MaxRecvPackSize uint16
MaxSendPackSize uint16
iReciver ITcpSocketServerReciver
nodelay bool
}
type MsgBasePack struct {
PackSize uint16
PackType uint16
Body []byte
StartTime time.Time
}
func (slf *TcpSocketServer) Register(listenAddr string,iReciver ITcpSocketServerReciver){
slf.listenAddr = listenAddr
slf.iReciver = iReciver
}
func (slf *TcpSocketServer) Start(){
slf.MaxRecvPackSize = 2048
slf.MaxSendPackSize = 40960
util.Go(slf.listenServer)
}
func (slf *TcpSocketServer) listenServer(){
slf.nodelay = true
listener, err := net.Listen("tcp", slf.listenAddr)
if err != nil {
service.GetLogger().Printf(service.LEVER_FATAL, "TcpSocketServer Listen error %+v",err)
os.Exit(1)
}
var clientId uint64
for {
conn, aerr := listener.Accept()
if aerr != nil {
service.GetLogger().Printf(service.LEVER_FATAL, "TcpSocketServer accept error %+v",aerr)
continue
}
if slf.nodelay {
//conn.(ifaceSetNoDelay)
}
for {
clientId += 1
if slf.mapClient.Get(clientId)!= nil {
continue
}
sc :=&SClient{id:clientId,conn:conn,tcpserver:slf,remoteip:conn.RemoteAddr().String(),starttime:time.Now().UnixNano(),
recvPack:util.NewSyncQueue(),sendPack:util.NewSyncQueue()}
slf.mapClient.Set(clientId,sc)
util.Go(sc.listendata)
//收来自客户端数据
util.Go(sc.onrecv)
//发送数据队列
util.Go(sc.onsend)
break
}
}
}
func (slf *TcpSocketServer) Close(clientid uint64) error {
pClient := slf.mapClient.Get(clientid)
if pClient == nil {
return fmt.Errorf("clientid %d is not in connect pool.",clientid)
}
pClient.(*SClient).Close()
return nil
}
func (slf *TcpSocketServer) SendMsg(clientid uint64,packtype uint16,message proto.Message) error{
pClient := slf.mapClient.Get(clientid)
if pClient == nil {
return fmt.Errorf("clientid %d is not in connect pool.",clientid)
}
return pClient.(*SClient).SendMsg(packtype,message)
}
func (slf *TcpSocketServer) Send(clientid uint64,pack *MsgBasePack) error{
pClient := slf.mapClient.Get(clientid)
if pClient == nil {
return fmt.Errorf("clientid %d is not in connect pool.",clientid)
}
return pClient.(*SClient).Send(pack)
}
func (slf *SClient) listendata(){
defer func() {
slf.Close()
slf.tcpserver.mapClient.Del(slf.id)
slf.tcpserver.iReciver.OnDisconnect(slf)
service.GetLogger().Printf(service.LEVER_DEBUG, "clent id %d return listendata...",slf.id)
}()
slf.tcpserver.iReciver.OnConnected(slf)
//获取一个连接的reader读取流
reader := bufio.NewReader(slf.conn)
//临时接受数据的buff
var buff []byte //tmprecvbuf
var tmpbuff []byte
var buffDataSize uint16
tmpbuff = make([]byte,2048)
//解析包数据
var pack MsgBasePack
for {
n,err := reader.Read(tmpbuff)
if err != nil || err == io.EOF {
service.GetLogger().Printf(service.LEVER_INFO, "clent id %d is disconnect %+v",slf.id,err)
return
}
buff = append(buff,tmpbuff[:n]...)
buffDataSize += uint16(n)
if buffDataSize> slf.tcpserver.MaxRecvPackSize {
service.GetLogger().Print(service.LEVER_WARN,"recv client id %d data size %d is over %d",slf.id,buffDataSize,slf.tcpserver.MaxRecvPackSize)
return
}
fillsize,bfillRet,fillhead := pack.FillData(buff,buffDataSize)
//提交校验头
if fillhead == true {
if pack.PackSize>slf.tcpserver.MaxRecvPackSize {
service.GetLogger().Printf(service.LEVER_WARN, "VerifyPackType error clent id %d is disconnect %d,%d",slf.id,pack.PackType, pack.PackSize)
return
}
}
if bfillRet == true {
slf.recvPack.Push(pack)
pack = MsgBasePack{}
}
if fillsize>0 {
buff = append(buff[fillsize:])
buffDataSize -= fillsize
}
}
}
func (slf *MsgBasePack) Bytes() []byte{
var bRet []byte
bRet = make([]byte,4)
binary.BigEndian.PutUint16(bRet,slf.PackSize)
binary.BigEndian.PutUint16(bRet[2:],slf.PackType)
bRet = append(bRet,slf.Body...)
return bRet
}
//返回值：填充多少字节，是否完成,是否填充头
func (slf *MsgBasePack) FillData(bdata []byte,datasize uint16) (uint16,bool,bool) {
var fillsize uint16
fillhead := false
//解包头
if slf.PackSize ==0 {
if datasize < 4 {
return 0,false,fillhead
}
slf.PackSize= binary.BigEndian.Uint16(bdata[:2])
slf.PackType= binary.BigEndian.Uint16(bdata[2:4])
fillsize += 4
fillhead = true
}
//解包体
if slf.PackSize>0 && datasize+4>=slf.PackSize {
slf.Body = append(slf.Body, bdata[fillsize:slf.PackSize]...)
fillsize += (slf.PackSize - fillsize)
return fillsize,true,fillhead
}
return fillsize,false,fillhead
}
func (slf *MsgBasePack) Clear() {
}
func (slf *MsgBasePack) Make(packtype uint16,data []byte) {
slf.PackType = packtype
slf.Body = data
slf.PackSize = uint16(unsafe.Sizeof(slf.PackType)*2)+uint16(len(data))
}
func (slf *SClient) Send(pack *MsgBasePack) error {
if slf.bClose == true {
return fmt.Errorf("client id %d is close!",slf.id)
}
slf.sendPack.Push(pack)
return nil
}
func (slf *SClient) SendMsg(packtype uint16,message proto.Message) error{
if slf.bClose == true {
return fmt.Errorf("client id %d is close!",slf.id)
}
var msg MsgBasePack
data,err := proto.Marshal(message)
if err != nil {
return err
}
msg.Make(packtype,data)
slf.sendPack.Push(&msg)
return nil
}
func (slf *SClient) onsend(){
defer func() {
slf.Close()
service.GetLogger().Printf(service.LEVER_DEBUG, "clent id %d return onsend...",slf.id)
}()
for {
pack,ok := slf.sendPack.TryPop()
if slf.bClose == true {
break
}
if ok == false || pack == nil {
time.Sleep(time.Millisecond*1)
continue
}
pPackData := pack.(*MsgBasePack)
_,e := slf.conn.Write(pPackData.Bytes())
if e!=nil {
service.GetLogger().Printf(service.LEVER_DEBUG, "clent id %d write error...",slf.id)
return
}
//fmt.Print("xxxxxxxxxxxxxxx:",n,e)
}
}
func (slf *SClient) onrecv(){
defer func() {
slf.Close()
service.GetLogger().Printf(service.LEVER_DEBUG, "clent id %d return onrecv...",slf.id)
}()
for {
pack,ok := slf.recvPack.TryPop()
if slf.bClose == true {
break
}
if ok == false || pack == nil {
time.Sleep(time.Millisecond*1)
continue
}
pMsg := pack.(MsgBasePack)
slf.tcpserver.iReciver.OnRecvMsg(slf,&pMsg)
}
}
func (slf *SClient) Close(){
if slf.bClose == false {
slf.conn.Close()
slf.bClose = true
slf.recvPack.Close()
slf.sendPack.Close()
}
}
func (slf *SClient) GetId() uint64{
return slf.id
}

259
network/websocketclient.go
View File

@@ -1,259 +0,0 @@
package network
import (
"errors"
"fmt"
"net/http"
"net/url"
"runtime/debug"
"github.com/duanhf2012/origin/service"
"github.com/duanhf2012/origin/sysmodule"
"github.com/gorilla/websocket"
"time"
)
//IWebsocketClient ...
type IWebsocketClient interface {
Init(slf IWebsocketClient, strurl, strProxyPath string, timeoutsec time.Duration) error
Start() error
WriteMessage(msg []byte) error
OnDisconnect() error
OnConnected() error
OnReadMessage(msg []byte) error
ReConnect()
}
//WebsocketClient ...
type WebsocketClient struct {
WsDailer *websocket.Dialer
conn *websocket.Conn
url string
state int //0未连接状态 1正在重连 2连接状态
bwritemsg chan []byte
closer chan bool
slf IWebsocketClient
timeoutsec time.Duration
bRun bool
ping []byte
}
const (
MAX_WRITE_MSG = 10240
)
//Init ...
func (ws *WebsocketClient) Init(slf IWebsocketClient, strurl, strProxyPath string, timeoutsec time.Duration) error {
ws.timeoutsec = timeoutsec
ws.slf = slf
if strProxyPath != "" {
proxy := func(_ *http.Request) (*url.URL, error) {
return url.Parse(strProxyPath)
}
if timeoutsec > 0 {
tosec := timeoutsec * time.Second
ws.WsDailer = &websocket.Dialer{Proxy: proxy, HandshakeTimeout: tosec}
} else {
ws.WsDailer = &websocket.Dialer{Proxy: proxy}
}
} else {
if timeoutsec > 0 {
tosec := timeoutsec * time.Second
ws.WsDailer = &websocket.Dialer{HandshakeTimeout: tosec}
} else {
ws.WsDailer = &websocket.Dialer{}
}
}
ws.url = strurl
ws.ping = []byte(`ping`)
return nil
}
func (ws *WebsocketClient) SetPing(ping string) {
ws.ping = []byte(ping)
}
//OnRun ...
func (ws *WebsocketClient) OnRun() error {
defer func() {
if r := recover(); r != nil {
coreInfo := string(debug.Stack())
coreInfo += "\n" + fmt.Sprintf("Core WebsocketClient url is %s. Core information is %v\n", ws.url, r)
service.GetLogger().Printf(service.LEVER_FATAL, coreInfo)
go ws.OnRun()
}
}()
for {
if ws.bRun == false {
break
}
if ws.state == 0 {
time.Sleep(2 * time.Second)
ws.StartConnect()
} else if ws.state == 1 {
ws.state = 0
close(ws.closer)
ws.conn.Close()
ws.slf.OnDisconnect()
} else if ws.state == 2 {
ws.conn.SetReadDeadline(time.Now().Add(ws.timeoutsec * time.Second))
_, message, err := ws.conn.ReadMessage()
if err != nil {
service.GetLogger().Printf(service.LEVER_WARN, "websocket client is disconnect [%s],information is %v", ws.url, err)
ws.conn.Close()
ws.state = 0
close(ws.closer)
ws.slf.OnDisconnect()
continue
}
ws.slf.OnReadMessage(message)
}
}
return nil
}
//StartConnect ...
func (ws *WebsocketClient) StartConnect() error {
var err error
ws.conn, _, err = ws.WsDailer.Dial(ws.url, nil)
service.GetLogger().Printf(sysmodule.LEVER_INFO, "connecting %s, %+v\n", ws.url, err)
if err != nil {
return err
}
ws.closer = make(chan bool)
ws.bwritemsg = make(chan []byte, MAX_WRITE_MSG)
ws.state = 2
ws.slf.OnConnected()
return nil
}
//Start ...
func (ws *WebsocketClient) Start() error {
if ws.bRun == false {
ws.bRun = true
ws.state = 0
go ws.OnRun()
go ws.writeMsg()
}
return nil
}
//触发
func (ws *WebsocketClient) writeMsg() error {
//dump处理
defer func() {
if r := recover(); r != nil {
coreInfo := string(debug.Stack())
coreInfo += "\n" + fmt.Sprintf("Core WebsocketClient url is %s. Core information is %v\n", ws.url, r)
service.GetLogger().Printf(service.LEVER_FATAL, coreInfo)
go ws.writeMsg()
}
}()
timerC := time.NewTicker(time.Second * 5).C
for {
if ws.bRun == false {
break
}
if ws.state == 0 {
time.Sleep(1 * time.Second)
continue
}
select {
case _, ok := <-ws.closer:
if ok == false {
break
}
case <-timerC:
if ws.state == 2 {
err := ws.WriteMessage(ws.ping)
if err != nil {
service.GetLogger().Printf(service.LEVER_WARN, "websocket client is disconnect [%s],information is %v", ws.url, err)
ws.state = 0
ws.conn.Close()
ws.slf.OnDisconnect()
}
}
case msg, ok := <-ws.bwritemsg:
if ok == true && ws.state == 2 {
ws.conn.SetWriteDeadline(time.Now().Add(ws.timeoutsec * time.Second))
err := ws.conn.WriteMessage(websocket.TextMessage, msg)
if err != nil {
service.GetLogger().Printf(service.LEVER_WARN, "websocket client is disconnect [%s],information is %v", ws.url, err)
ws.state = 0
ws.conn.Close()
ws.slf.OnDisconnect()
}
}
}
}
return nil
}
//ReConnect ...
func (ws *WebsocketClient) ReConnect() {
ws.state = 1
}
//WriteMessage ...
func (ws *WebsocketClient) WriteMessage(msg []byte) error {
if ws.closer == nil || ws.bwritemsg == nil {
service.GetLogger().Printf(service.LEVER_WARN, "WriteMessage data fail,websocket client is disconnect.")
return errors.New("riteMessage data fail,websocket client is disconnect.")
}
select {
case <-ws.closer:
service.GetLogger().Printf(service.LEVER_WARN, "WriteMessage data fail,websocket client is disconnect.")
return errors.New("riteMessage data fail,websocket client is disconnect.")
default:
if len(ws.bwritemsg) < MAX_WRITE_MSG {
ws.bwritemsg <- msg
} else {
service.GetLogger().Printf(service.LEVER_ERROR, "WriteMessage data fail,bwriteMsg is overload.")
return errors.New("WriteMessage data fail,bwriteMsg is overload.")
}
}
return nil
}
//OnDisconnect ...
func (ws *WebsocketClient) OnDisconnect() error {
return nil
}
//OnConnected ...
func (ws *WebsocketClient) OnConnected() error {
return nil
}
//OnReadMessage 触发
func (ws *WebsocketClient) OnReadMessage(msg []byte) error {
return nil
}
//Stop ...
func (ws *WebsocketClient) Stop() {
ws.bRun = false
}

328
network/websocketserver.go
View File

@@ -1,328 +0,0 @@
package network
import (
"crypto/tls"
"errors"
"fmt"
"github.com/duanhf2012/origin/util"
"net/http"
"os"
"runtime/debug"
"sync"
"time"
"github.com/duanhf2012/origin/service"
"github.com/duanhf2012/origin/sysmodule"
"github.com/gorilla/mux"
"github.com/gorilla/websocket"
"github.com/gotoxu/cors"
)
type IWebsocketServer interface {
SendMsg(clientid uint64, messageType int, msg []byte) bool
CreateClient(conn *websocket.Conn) *WSClient
Disconnect(clientid uint64)
ReleaseClient(pclient *WSClient)
Clients() []uint64
BroadcastMsg(messageType int, msg []byte) int
}
type IMessageReceiver interface {
initReciver(messageReciver IMessageReceiver, websocketServer IWebsocketServer)
OnConnected(clientid uint64)
OnDisconnect(clientid uint64, err error)
OnRecvMsg(clientid uint64, msgtype int, data []byte)
OnHandleHttp(w http.ResponseWriter, r *http.Request)
IsInit() bool
}
type Reciver struct {
messageReciver IMessageReceiver
bEnableCompression bool
}
type BaseMessageReciver struct {
messageReciver IMessageReceiver
WsServer IWebsocketServer
}
type WSClient struct {
clientid uint64
conn *websocket.Conn
bwritemsg chan WSMessage
}
type WSMessage struct {
msgtype int
bwritemsg []byte
}
type WebsocketServer struct {
wsUri string
maxClientid uint64 //记录当前最新clientid
mapClient map[uint64]*WSClient
locker sync.RWMutex
port uint16
httpserver *http.Server
reciver map[string]Reciver
caList []CA
iswss bool
}
const (
MAX_MSG_COUNT = 20480
)
func (slf *WebsocketServer) Init(port uint16) {
slf.port = port
slf.mapClient = make(map[uint64]*WSClient)
}
func (slf *WebsocketServer) CreateClient(conn *websocket.Conn) *WSClient {
slf.locker.Lock()
slf.maxClientid++
clientid := slf.maxClientid
pclient := &WSClient{clientid, conn, make(chan WSMessage, MAX_MSG_COUNT+1)}
slf.mapClient[pclient.clientid] = pclient
slf.locker.Unlock()
service.GetLogger().Printf(sysmodule.LEVER_INFO, "Client id %d is connected.", clientid)
return pclient
}
func (slf *WebsocketServer) ReleaseClient(pclient *WSClient) {
pclient.conn.Close()
slf.locker.Lock()
delete(slf.mapClient, pclient.clientid)
slf.locker.Unlock()
//关闭写管道
close(pclient.bwritemsg)
service.GetLogger().Printf(sysmodule.LEVER_INFO, "Client id %d is disconnected.", pclient.clientid)
}
func (slf *WebsocketServer) SetupReciver(pattern string, messageReciver IMessageReceiver, bEnableCompression bool) {
messageReciver.initReciver(messageReciver, slf)
if slf.reciver == nil {
slf.reciver = make(map[string]Reciver)
}
slf.reciver[pattern] = Reciver{messageReciver, bEnableCompression}
}
func (slf *WebsocketServer) startListen() {
listenPort := fmt.Sprintf(":%d", slf.port)
var tlscatList []tls.Certificate
var tlsConfig *tls.Config
for _, cadata := range slf.caList {
cer, err := tls.LoadX509KeyPair(cadata.certfile, cadata.keyfile)
if err != nil {
service.GetLogger().Printf(sysmodule.LEVER_FATAL, "load CA %s-%s file is error :%s", cadata.certfile, cadata.keyfile, err.Error())
os.Exit(1)
return
}
tlscatList = append(tlscatList, cer)
}
if len(tlscatList) > 0 {
tlsConfig = &tls.Config{Certificates: tlscatList}
}
slf.httpserver = &http.Server{
Addr: listenPort,
Handler: slf.initRouterHandler(),
ReadTimeout: 10 * time.Second,
WriteTimeout: 10 * time.Second,
MaxHeaderBytes: 1 << 20,
TLSConfig: tlsConfig,
}
var err error
if slf.iswss == true {
err = slf.httpserver.ListenAndServeTLS("", "")
} else {
err = slf.httpserver.ListenAndServe()
}
if err != nil {
service.GetLogger().Printf(sysmodule.LEVER_FATAL, "http.ListenAndServe(%d, nil) error:%v\n", slf.port, err)
os.Exit(1)
}
}
func (slf *WSClient) startSendMsg() {
for {
msgbuf, ok := <-slf.bwritemsg
if ok == false {
break
}
slf.conn.SetWriteDeadline(time.Now().Add(15 * time.Second))
err := slf.conn.WriteMessage(msgbuf.msgtype, msgbuf.bwritemsg)
if err != nil {
service.GetLogger().Printf(sysmodule.LEVER_INFO, "write client id %d is error :%v\n", slf.clientid, err)
break
}
}
}
func (slf *WebsocketServer) Start() {
go slf.startListen()
}
func (slf *WebsocketServer) Clients() []uint64 {
slf.locker.RLock()
defer slf.locker.RUnlock()
r := make([]uint64, 0, len(slf.mapClient))
for i, _ := range slf.mapClient {
r = append(r, i)
}
return r
}
func (slf *WebsocketServer) BroadcastMsg(messageType int, msg []byte) int {
slf.locker.RLock()
defer slf.locker.RUnlock()
err := 0
wsMsg := WSMessage{messageType, msg}
for _, value := range slf.mapClient {
if len(value.bwritemsg) >= MAX_MSG_COUNT {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "message chan is full :%d\n", len(value.bwritemsg))
err++
}
value.bwritemsg <- wsMsg
}
return err
}
func (slf *WebsocketServer) SendMsg(clientid uint64, messageType int, msg []byte) bool {
slf.locker.RLock()
defer slf.locker.RUnlock()
value, ok := slf.mapClient[clientid]
if ok == false {
return false
}
if len(value.bwritemsg) >= MAX_MSG_COUNT {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "message chan is full :%d\n", len(value.bwritemsg))
return false
}
value.bwritemsg <- WSMessage{messageType, msg}
return true
}
func (slf *WebsocketServer) Disconnect(clientid uint64) {
slf.locker.Lock()
defer slf.locker.Unlock()
value, ok := slf.mapClient[clientid]
if ok == false {
return
}
value.conn.Close()
}
func (slf *WebsocketServer) Stop() {
}
func (slf *BaseMessageReciver) startReadMsg(pclient *WSClient) {
defer func() {
if r := recover(); r != nil {
var coreInfo string
coreInfo = string(debug.Stack())
coreInfo += "\n" + fmt.Sprintf("Core information is %v\n", r)
service.GetLogger().Printf(service.LEVER_FATAL, coreInfo)
slf.messageReciver.OnDisconnect(pclient.clientid, errors.New("Core dump"))
slf.WsServer.ReleaseClient(pclient)
}
}()
var maxTimeStamp int64
var maxMsgType int
logMinMsgTime :=time.Millisecond*300
statisticsIntervalTm := util.Timer{}
statisticsIntervalTm.SetupTimer(1000 * 15)//15秒间隔
for {
pclient.conn.SetReadDeadline(time.Now().Add(15 * time.Second))
msgtype, message, err := pclient.conn.ReadMessage()
if err != nil {
slf.messageReciver.OnDisconnect(pclient.clientid, err)
slf.WsServer.ReleaseClient(pclient)
return
}
if statisticsIntervalTm.CheckTimeOut() {
service.GetLogger().Printf(service.LEVER_INFO, "MaxMsgtype:%d,diff:%d",maxMsgType,maxTimeStamp)
}
//记录处理时间
startRecvTm := time.Now().UnixNano()
slf.messageReciver.OnRecvMsg(pclient.clientid, msgtype, message)
diff := time.Now().UnixNano() - startRecvTm
if diff> maxTimeStamp{
maxTimeStamp = diff
maxMsgType = msgtype
}
if diff >= int64(logMinMsgTime) {
service.GetLogger().Printf(service.LEVER_WARN, "Process slowly MaxMsgtype:%d,diff:%d",maxMsgType,maxTimeStamp)
}
}
}
func (slf *BaseMessageReciver) initReciver(messageReciver IMessageReceiver, websocketServer IWebsocketServer) {
slf.messageReciver = messageReciver
slf.WsServer = websocketServer
}
func (slf *BaseMessageReciver) OnConnected(clientid uint64) {
}
func (slf *BaseMessageReciver) OnDisconnect(clientid uint64, err error) {
}
func (slf *BaseMessageReciver) OnRecvMsg(clientid uint64, msgtype int, data []byte) {
}
func (slf *BaseMessageReciver) OnHandleHttp(w http.ResponseWriter, r *http.Request) {
conn, err := websocket.Upgrade(w, r, w.Header(), 1024, 1024)
if err != nil {
http.Error(w, "Could not open websocket connection", http.StatusBadRequest)
return
}
pclient := slf.WsServer.CreateClient(conn)
slf.messageReciver.OnConnected(pclient.clientid)
go pclient.startSendMsg()
go slf.startReadMsg(pclient)
}
func (slf *WebsocketServer) initRouterHandler() http.Handler {
r := mux.NewRouter()
for pattern, reciver := range slf.reciver {
if reciver.messageReciver.IsInit() == true {
r.HandleFunc(pattern, reciver.messageReciver.OnHandleHttp)
}
}
cors := cors.AllowAll()
return cors.Handler(r)
}
func (slf *WebsocketServer) SetWSS(certfile string, keyfile string) bool {
if certfile == "" || keyfile == "" {
return false
}
slf.caList = append(slf.caList, CA{certfile, keyfile})
slf.iswss = true
return true
}

323
network/wsagentserver.go
View File

@@ -1,323 +0,0 @@
package network
import (
"crypto/tls"
"errors"
"fmt"
"net/http"
"os"
"reflect"
"runtime/debug"
"sync"
"time"
"github.com/duanhf2012/origin/service"
"github.com/duanhf2012/origin/sysmodule"
"github.com/gotoxu/cors"
"github.com/gorilla/mux"
"github.com/gorilla/websocket"
)
type IWSAgentServer interface {
SendMsg(agentid uint32, messageType int, msg []byte) bool
CreateAgent(urlPattern string, conn *websocket.Conn) IAgent
Disconnect(agentid uint32)
ReleaseAgent(iagent IAgent)
}
type IAgent interface {
initAgent(conn *websocket.Conn, agentid uint32, iagent IAgent, WSAgentServer IWSAgentServer)
startReadMsg()
startSendMsg()
OnConnected()
OnDisconnect(err error)
OnRecvMsg(msgtype int, data []byte)
//OnHandleHttp(w http.ResponseWriter, r *http.Request)
GetAgentId() uint32
getConn() *websocket.Conn
getWriteMsgChan() chan WSAgentMessage
}
type BaseAgent struct {
service.BaseModule
WsServer IWSAgentServer
agent IAgent
agentid uint32
conn *websocket.Conn
bwritemsg chan WSAgentMessage
iagent IAgent
}
type WSAgentMessage struct {
msgtype int
bwritemsg []byte
}
type WSAgentServer struct {
service.BaseModule
wsUri string
maxAgentid uint32 //记录当前最新agentid
//mapAgent map[uint32]IAgent
locker sync.Mutex
port uint16
httpserver *http.Server
regAgent map[string]reflect.Type
caList []CA
iswss bool
}
const (
MAX_AGENT_MSG_COUNT = 20480
)
func (slf *WSAgentServer) Init(port uint16) {
slf.port = port
}
func (slf *WSAgentServer) CreateAgent(urlPattern string, conn *websocket.Conn) IAgent {
slf.locker.Lock()
iAgent, ok := slf.regAgent[urlPattern]
if ok == false {
slf.locker.Unlock()
service.GetLogger().Printf(sysmodule.LEVER_WARN, "Cannot find %s pattern!", urlPattern)
return nil
}
v := reflect.New(iAgent).Elem().Addr().Interface()
if v == nil {
slf.locker.Unlock()
service.GetLogger().Printf(sysmodule.LEVER_WARN, "new %s pattern agent type is error!", urlPattern)
return nil
}
pModule := v.(service.IModule)
iagent := v.(IAgent)
slf.maxAgentid++
agentid := slf.maxAgentid
iagent.initAgent(conn, agentid, iagent, slf)
slf.AddModule(pModule)
slf.locker.Unlock()
service.GetLogger().Printf(sysmodule.LEVER_INFO, "Agent id %d is connected.", iagent.GetAgentId())
return iagent
}
func (slf *WSAgentServer) ReleaseAgent(iagent IAgent) {
iagent.getConn().Close()
slf.locker.Lock()
slf.ReleaseModule(iagent.GetAgentId())
//delete(slf.mapAgent, iagent.GetAgentId())
slf.locker.Unlock()
//关闭写管道
close(iagent.getWriteMsgChan())
service.GetLogger().Printf(sysmodule.LEVER_INFO, "Agent id %d is disconnected.", iagent.GetAgentId())
}
func (slf *WSAgentServer) SetupAgent(pattern string, agent IAgent, bEnableCompression bool) {
if slf.regAgent == nil {
slf.regAgent = make(map[string]reflect.Type)
}
slf.regAgent[pattern] = reflect.TypeOf(agent).Elem() //reflect.TypeOf(agent).Elem()
}
func (slf *WSAgentServer) startListen() {
listenPort := fmt.Sprintf(":%d", slf.port)
var tlscatList []tls.Certificate
var tlsConfig *tls.Config
for _, cadata := range slf.caList {
cer, err := tls.LoadX509KeyPair(cadata.certfile, cadata.keyfile)
if err != nil {
service.GetLogger().Printf(sysmodule.LEVER_FATAL, "load CA %s-%s file is error :%s", cadata.certfile, cadata.keyfile, err.Error())
os.Exit(1)
return
}
tlscatList = append(tlscatList, cer)
}
if len(tlscatList) > 0 {
tlsConfig = &tls.Config{Certificates: tlscatList}
}
slf.httpserver = &http.Server{
Addr: listenPort,
Handler: slf.initRouterHandler(),
ReadTimeout: 10 * time.Second,
WriteTimeout: 10 * time.Second,
MaxHeaderBytes: 1 << 20,
TLSConfig: tlsConfig,
}
var err error
if slf.iswss == true {
err = slf.httpserver.ListenAndServeTLS("", "")
} else {
err = slf.httpserver.ListenAndServe()
}
if err != nil {
service.GetLogger().Printf(sysmodule.LEVER_FATAL, "http.ListenAndServe(%d, nil) error:%v\n", slf.port, err)
os.Exit(1)
}
}
func (slf *BaseAgent) startSendMsg() {
for {
msgbuf, ok := <-slf.bwritemsg
if ok == false {
break
}
slf.conn.SetWriteDeadline(time.Now().Add(15 * time.Second))
err := slf.conn.WriteMessage(msgbuf.msgtype, msgbuf.bwritemsg)
if err != nil {
service.GetLogger().Printf(sysmodule.LEVER_INFO, "write agent id %d is error :%v\n", slf.GetAgentId(), err)
break
}
}
}
func (slf *WSAgentServer) Start() {
go slf.startListen()
}
func (slf *WSAgentServer) GetAgentById(agentid uint32) IAgent {
pModule := slf.GetModuleById(agentid)
if pModule == nil {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "GetAgentById :%d is fail.\n", agentid)
return nil
}
return pModule.(IAgent)
}
func (slf *WSAgentServer) SendMsg(agentid uint32, messageType int, msg []byte) bool {
slf.locker.Lock()
defer slf.locker.Unlock()
iagent := slf.GetAgentById(agentid)
if iagent == nil {
return false
}
if len(iagent.getWriteMsgChan()) >= MAX_AGENT_MSG_COUNT {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "message chan is full :%d\n", len(iagent.getWriteMsgChan()))
return false
}
iagent.getWriteMsgChan() <- WSAgentMessage{messageType, msg}
return true
}
func (slf *WSAgentServer) Disconnect(agentid uint32) {
slf.locker.Lock()
defer slf.locker.Unlock()
iagent := slf.GetAgentById(agentid)
if iagent == nil {
return
}
iagent.getConn().Close()
}
func (slf *WSAgentServer) Stop() {
}
func (slf *BaseAgent) startReadMsg() {
defer func() {
if r := recover(); r != nil {
var coreInfo string
coreInfo = string(debug.Stack())
coreInfo += "\n" + fmt.Sprintf("Core information is %v\n", r)
service.GetLogger().Printf(service.LEVER_FATAL, coreInfo)
slf.agent.OnDisconnect(errors.New("Core dump"))
slf.WsServer.ReleaseAgent(slf.agent)
}
}()
slf.agent.OnConnected()
for {
slf.conn.SetReadDeadline(time.Now().Add(15 * time.Second))
msgtype, message, err := slf.conn.ReadMessage()
if err != nil {
slf.agent.OnDisconnect(err)
slf.WsServer.ReleaseAgent(slf.agent)
return
}
slf.agent.OnRecvMsg(msgtype, message)
}
}
func (slf *WSAgentServer) initRouterHandler() http.Handler {
r := mux.NewRouter()
for pattern, _ := range slf.regAgent {
r.HandleFunc(pattern, slf.OnHandleHttp)
}
cors := cors.AllowAll()
return cors.Handler(r)
}
func (slf *WSAgentServer) SetWSS(certfile string, keyfile string) bool {
if certfile == "" || keyfile == "" {
return false
}
slf.caList = append(slf.caList, CA{certfile, keyfile})
slf.iswss = true
return true
}
func (slf *BaseAgent) GetAgentId() uint32 {
return slf.agentid
}
func (slf *BaseAgent) initAgent(conn *websocket.Conn, agentid uint32, iagent IAgent, WSAgentServer IWSAgentServer) {
slf.agent = iagent
slf.WsServer = WSAgentServer
slf.bwritemsg = make(chan WSAgentMessage, MAX_AGENT_MSG_COUNT)
slf.agentid = agentid
slf.conn = conn
}
func (slf *BaseAgent) OnConnected() {
}
func (slf *BaseAgent) OnDisconnect(err error) {
}
func (slf *BaseAgent) OnRecvMsg(msgtype int, data []byte) {
}
func (slf *BaseAgent) getConn() *websocket.Conn {
return slf.conn
}
func (slf *BaseAgent) getWriteMsgChan() chan WSAgentMessage {
return slf.bwritemsg
}
func (slf *BaseAgent) SendMsg(agentid uint32, messageType int, msg []byte) bool {
return slf.WsServer.SendMsg(agentid, messageType, msg)
}
func (slf *WSAgentServer) OnHandleHttp(w http.ResponseWriter, r *http.Request) {
conn, err := websocket.Upgrade(w, r, w.Header(), 1024, 1024)
if err != nil {
http.Error(w, "Could not open websocket connection!", http.StatusBadRequest)
return
}
agent := slf.CreateAgent(r.URL.Path, conn)
fmt.Print(agent.GetAgentId())
slf.AddModule(agent.(service.IModule))
go agent.startSendMsg()
go agent.startReadMsg()
}

115
node/node.go Normal file
View File

@@ -0,0 +1,115 @@
package node
import (
"fmt"
"github.com/duanhf2012/originnet/service"
"github.com/duanhf2012/originnet/cluster"
"io/ioutil"
"os"
"os/signal"
"strconv"
"syscall"
"time"
)
var closeSig chan bool
var sigs chan os.Signal
var preSetupService []service.IService //预安装
func init() {
closeSig = make(chan bool,1)
sigs = make(chan os.Signal, 3)
signal.Notify(sigs, syscall.SIGINT, syscall.SIGTERM,syscall.Signal(10))
}
func getRunProcessPid() (int,error) {
f, err := os.OpenFile(os.Args[0]+".pid", os.O_RDONLY, 0600)
defer f.Close()
if err!= nil {
return 0,err
}
pidbyte,errs := ioutil.ReadAll(f)
if errs!=nil {
return 0,errs
}
return strconv.Atoi(string(pidbyte))
}
func writeProcessPid() {
//pid
f, err := os.OpenFile(os.Args[0]+".pid", os.O_WRONLY|os.O_TRUNC|os.O_CREATE, 0600)
defer f.Close()
if err != nil {
fmt.Println(err.Error())
os.Exit(-1)
} else {
_,err=f.Write([]byte(fmt.Sprintf("%d",os.Getpid())))
if err != nil {
fmt.Println(err.Error())
os.Exit(-1)
}
}
}
func GetNodeId() int {
return 1
}
func Init(){
//1.初始化集群
err := cluster.GetCluster().Init(GetNodeId())
if err != nil {
panic(err)
}
//2.service模块初始化
service.Init(closeSig)
//3.初始化预安装的服务
for _,s := range preSetupService {
pServiceCfg := cluster.GetCluster().GetServiceCfg(s.GetName())
s.Init(s,cluster.GetRpcClient,cluster.GetRpcServer,pServiceCfg)
//是否配置的service
if cluster.GetCluster().IsConfigService(s.GetName()) == false {
continue
}
service.Setup(s)
}
}
func Start() {
cluster.GetCluster().Start()
service.Start()
writeProcessPid()
for {
select {
case <-sigs:
fmt.Printf("Recv stop sig")
break
default:
time.Sleep(time.Second)
}
}
close(closeSig)
service.WaitStop()
}
func Setup(s ...service.IService) {
for _,sv := range s {
preSetupService = append(preSetupService,sv)
}
}
func GetService(servicename string) service.IService {
return service.GetService(servicename)
}
func SetConfigDir(configdir string){
cluster.SetConfigDir(configdir)
}

175
originnode/node.go
View File

@@ -1,175 +0,0 @@
package originnode
import (
"fmt"
"log"
"strconv"
"strings"
"time"
"github.com/duanhf2012/origin/util"
"net/http"
_ "net/http/pprof"
"os"
"os/signal"
"sync"
"syscall"
"github.com/duanhf2012/origin/cluster"
"github.com/duanhf2012/origin/service"
"github.com/duanhf2012/origin/sysmodule"
"github.com/duanhf2012/origin/sysservice"
)
type CExitCtl struct {
exitChan chan bool
waitGroup *sync.WaitGroup
}
type COriginNode struct {
CExitCtl
serviceManager service.IServiceManager
sigs chan os.Signal
debugListenAddress string
}
var initservicelist []service.IService
func InitService(iservice service.IService) {
initservicelist = append(initservicelist, iservice)
}
func (s *COriginNode) Init() {
s.SetupService(initservicelist...)
//初始化全局模块
logger := service.InstanceServiceMgr().FindService("syslog").(service.ILogger)
ret := service.InstanceServiceMgr().Init(logger, s.exitChan, s.waitGroup)
if ret == false {
os.Exit(-1)
}
util.Log = logger.Printf
s.sigs = make(chan os.Signal, 1)
signal.Notify(s.sigs, syscall.SIGINT, syscall.SIGTERM)
}
// OpenDebugCheck ("localhost:6060")...http://localhost:6060/
func (s *COriginNode) OpenDebugCheck(listenAddress string) {
s.debugListenAddress = listenAddress
}
func (s *COriginNode) SetupService(services ...service.IService) {
ppService := &sysservice.PProfService{}
services = append(services, ppService)
cluster.InstanceClusterMgr().AddLocalService(ppService)
for i := 0; i < len(services); i++ {
services[i].Init(services[i])
if cluster.InstanceClusterMgr().HasLocalService(services[i].GetServiceName()) == true {
service.InstanceServiceMgr().Setup(services[i])
}
}
}
func (s *COriginNode) Start() {
if s.debugListenAddress != "" {
go func() {
log.Println(http.ListenAndServe(s.debugListenAddress, nil))
}()
}
//开始运行集群
cluster.InstanceClusterMgr().Start()
//开启所有服务
service.InstanceServiceMgr().Start()
//监听退出信号
select {
case <-s.sigs:
service.GetLogger().Printf(sysmodule.LEVER_WARN, "Recv stop sig")
fmt.Printf("Recv stop sig")
}
//停止运行程序
s.Stop()
service.GetLogger().Printf(sysmodule.LEVER_INFO, "Node stop run...")
}
func (s *COriginNode) Stop() {
close(s.exitChan)
s.waitGroup.Wait()
}
func GetCmdParamNodeId() int {
if len(os.Args) < 2 {
return 0
//return fmt.Errorf("Param error not find NodeId=number")
}
parts := strings.Split(os.Args[1], "=")
if len(parts) < 2 {
return 0
//return fmt.Errorf("Param error not find NodeId=number")
}
if parts[0] != "NodeId" {
return 0
//return fmt.Errorf("Param error not find NodeId=number")
}
//读取配置
currentNodeid, err := strconv.Atoi(parts[1])
if err != nil {
return 0
}
return currentNodeid
}
func NewOriginNode() *COriginNode {
CurrentNodeId := GetCmdParamNodeId()
if CurrentNodeId == 0 {
fmt.Print("Param error not find NodeId=number")
os.Exit(-1)
}
//创建模块
node := new(COriginNode)
node.exitChan = make(chan bool)
node.waitGroup = &sync.WaitGroup{}
//安装系统服务
syslogservice := &sysservice.LogService{}
syslogservice.InitLog("syslog", fmt.Sprintf("syslog_%d", CurrentNodeId), sysmodule.LEVER_DEBUG)
service.InstanceServiceMgr().Setup(syslogservice)
//初始化集群对象
err := cluster.InstanceClusterMgr().Init(CurrentNodeId)
if err != nil {
fmt.Print(err)
os.Exit(-1)
return nil
}
return node
}
func (s *COriginNode) GetSysLog() *sysservice.LogService {
logService := service.InstanceServiceMgr().FindService("syslog")
if logService == nil {
fmt.Printf("Cannot find syslog service!")
os.Exit(-1)
return nil
}
return logService.(*sysservice.LogService)
}
func (s *COriginNode) EnableMonitorModule(checkInterval time.Duration){
service.EnableDeadForMonitor(checkInterval)
}

514
rpc/client.go
View File

@@ -1,360 +1,208 @@
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package rpc
import (
"bufio"
"encoding/gob"
"errors"
"fmt"
"io"
"log"
"net"
"net/http"
"github.com/duanhf2012/originnet/log"
"github.com/duanhf2012/originnet/network"
"math"
"reflect"
"strings"
"sync"
"time"
)
// ServerError represents an error that has been returned from
// the remote side of the RPC connection.
type ServerError string
func (e ServerError) Error() string {
return string(e)
}
var ErrShutdown = errors.New("connection is shut down")
// Call represents an active RPC.
type Call struct {
ServiceMethod string // The name of the service and method to call.
Args interface{} // The argument to the function (*struct).
Reply interface{} // The reply from the function (*struct).
Error error // After completion, the error status.
Done chan *Call // Strobes when call is complete.
}
// Client represents an RPC Client.
// There may be multiple outstanding Calls associated
// with a single Client, and a Client may be used by
// multiple goroutines simultaneously.
type Client struct {
codec ClientCodec
blocalhost bool
network.TCPClient
conn *network.TCPConn
reqMutex sync.Mutex // protects following
request Request
mutex sync.Mutex // protects following
seq uint64
pending map[uint64]*Call
closing bool // user has called Close
shutdown bool // server has told us to stop
bPipe bool
pendingLock sync.RWMutex
startSeq uint64
pending map[uint64]*Call
}
// A ClientCodec implements writing of RPC requests and
// reading of RPC responses for the client side of an RPC session.
// The client calls WriteRequest to write a request to the connection
// and calls ReadResponseHeader and ReadResponseBody in pairs
// to read responses. The client calls Close when finished with the
// connection. ReadResponseBody may be called with a nil
// argument to force the body of the response to be read and then
// discarded.
// See NewClient's comment for information about concurrent access.
type ClientCodec interface {
WriteRequest(*Request, interface{}) error
ReadResponseHeader(*Response) error
ReadResponseBody(interface{}) error
Close() error
func (slf *Client) NewClientAgent(conn *network.TCPConn) network.Agent {
slf.conn = conn
return slf
}
func (client *Client) IsClosed() bool {
client.reqMutex.Lock()
defer client.reqMutex.Unlock()
return client.shutdown || client.closing
}
func (client *Client) send(call *Call, queueMode bool) {
client.reqMutex.Lock()
defer client.reqMutex.Unlock()
// Register this call.
client.mutex.Lock()
if client.shutdown || client.closing {
client.mutex.Unlock()
call.Error = ErrShutdown
call.done()
return
func (slf *Client) Connect(addr string) error {
slf.Addr = addr
if strings.Index(addr,"localhost") == 0 {
slf.blocalhost = true
return nil
}
seq := client.seq
client.seq++
client.pending[seq] = call
client.mutex.Unlock()
slf.ConnNum = 1
slf.ConnectInterval = time.Second*2
slf.PendingWriteNum = 10000
slf.AutoReconnect = true
slf.LenMsgLen =2
slf.MinMsgLen = 2
slf.MaxMsgLen = math.MaxUint16
slf.NewAgent =slf.NewClientAgent
slf.LittleEndian = LittleEndian
// Encode and send the request.
client.request.Seq = seq
client.request.ServiceMethod = call.ServiceMethod
client.request.QueueMode = queueMode
err := client.codec.WriteRequest(&client.request, call.Args)
if err != nil {
client.mutex.Lock()
call = client.pending[seq]
delete(client.pending, seq)
client.mutex.Unlock()
if call != nil {
call.Error = err
call.done()
}
slf.pendingLock.Lock()
for _,v := range slf.pending {
v.Err = fmt.Errorf("node is disconnect.")
v.done <- v
}
slf.pending = map[uint64]*Call{}
slf.pendingLock.Unlock()
slf.Start()
return nil
}
func (client *Client) input() {
var err error
var response Response
for err == nil {
response = Response{}
err = client.codec.ReadResponseHeader(&response)
if err != nil {
break
}
seq := response.Seq
client.mutex.Lock()
call := client.pending[seq]
delete(client.pending, seq)
client.mutex.Unlock()
switch {
case call == nil:
// We've got no pending call. That usually means that
// WriteRequest partially failed, and call was already
// removed; response is a server telling us about an
// error reading request body. We should still attempt
// to read error body, but there's no one to give it to.
err = client.codec.ReadResponseBody(nil)
if err != nil {
err = errors.New("reading error body: " + err.Error())
}
case response.Error != "":
// We've got an error response. Give this to the request;
// any subsequent requests will get the ReadResponseBody
// error if there is one.
call.Error = ServerError(response.Error)
err = client.codec.ReadResponseBody(nil)
if err != nil {
err = errors.New("reading error body: " + err.Error())
}
call.done()
default:
err = client.codec.ReadResponseBody(call.Reply)
if err != nil {
call.Error = errors.New("reading body " + err.Error())
}
if client.bPipe {
err = nil
}
call.done()
}
}
// Terminate pending calls.
client.reqMutex.Lock()
client.mutex.Lock()
client.shutdown = true
closing := client.closing
if err == io.EOF {
if closing {
err = ErrShutdown
} else {
err = io.ErrUnexpectedEOF
}
}
for _, call := range client.pending {
call.Error = err
call.done()
}
client.mutex.Unlock()
client.reqMutex.Unlock()
if debugLog && err != io.EOF && !closing {
log.Println("rpc: client protocol error:", err)
}
}
func (call *Call) done() {
select {
case call.Done <- call:
// ok
default:
// We don't want to block here. It is the caller's responsibility to make
// sure the channel has enough buffer space. See comment in Go().
if debugLog {
log.Println("rpc: discarding Call reply due to insufficient Done chan capacity")
}
}
}
// NewClient returns a new Client to handle requests to the
// set of services at the other end of the connection.
// It adds a buffer to the write side of the connection so
// the header and payload are sent as a unit.
//
// The read and write halves of the connection are serialized independently,
// so no interlocking is required. However each half may be accessed
// concurrently so the implementation of conn should protect against
// concurrent reads or concurrent writes.
func NewClient(conn io.ReadWriteCloser, isPipe bool) *Client {
encBuf := bufio.NewWriter(conn)
client := &gobClientCodec{conn, gob.NewDecoder(conn), gob.NewEncoder(encBuf), encBuf}
return NewClientWithCodec(client, isPipe)
}
// NewClientWithCodec is like NewClient but uses the specified
// codec to encode requests and decode responses.
func NewClientWithCodec(codec ClientCodec, isPipe bool) *Client {
client := &Client{
codec: codec,
pending: make(map[uint64]*Call),
bPipe: isPipe,
}
go client.input()
return client
}
type gobClientCodec struct {
rwc io.ReadWriteCloser
dec *gob.Decoder
enc *gob.Encoder
encBuf *bufio.Writer
}
func (c *gobClientCodec) WriteRequest(r *Request, body interface{}) (err error) {
if err = c.enc.Encode(r); err != nil {
return
}
if err = c.enc.Encode(body); err != nil {
return
}
return c.encBuf.Flush()
}
func (c *gobClientCodec) ReadResponseHeader(r *Response) error {
return c.dec.Decode(r)
}
func (c *gobClientCodec) ReadResponseBody(body interface{}) error {
return c.dec.Decode(body)
}
func (c *gobClientCodec) Close() error {
return c.rwc.Close()
}
// DialHTTP connects to an HTTP RPC server at the specified network address
// listening on the default HTTP RPC path.
func DialHTTP(network, address string) (*Client, error) {
return DialHTTPPath(network, address, DefaultRPCPath)
}
// DialHTTPPath connects to an HTTP RPC server
// at the specified network address and path.
func DialHTTPPath(network, address, path string) (*Client, error) {
var err error
conn, err := net.Dial(network, address)
if err != nil {
return nil, err
}
tcpconn, _ := conn.(*net.TCPConn)
tcpconn.SetNoDelay(true)
io.WriteString(conn, "CONNECT "+path+" HTTP/1.0\n\n")
// Require successful HTTP response
// before switching to RPC protocol.
resp, err := http.ReadResponse(bufio.NewReader(conn), &http.Request{Method: "CONNECT"})
if err == nil && resp.Status == connected {
return NewClient(conn, false), nil
}
if err == nil {
err = errors.New("unexpected HTTP response: " + resp.Status)
}
conn.Close()
return nil, &net.OpError{
Op: "dial-http",
Net: network + " " + address,
Addr: nil,
Err: err,
}
}
// Dial connects to an RPC server at the specified network address.
func Dial(network, address string) (*Client, error) {
conn, err := net.Dial(network, address)
if err != nil {
return nil, err
}
tcpconn, _ := conn.(*net.TCPConn)
tcpconn.SetNoDelay(true)
return NewClient(conn, false), nil
}
func DialTimeOut(network, address string, timeout time.Duration) (*Client, error) {
conn, err := net.DialTimeout(network, address, timeout)
if err != nil {
return nil, err
}
tcpconn, _ := conn.(*net.TCPConn)
tcpconn.SetNoDelay(true)
return NewClient(conn, false), nil
}
// Close calls the underlying codec's Close method. If the connection is already
// shutting down, ErrShutdown is returned.
func (client *Client) Close() error {
client.mutex.Lock()
if client.closing {
client.mutex.Unlock()
return ErrShutdown
}
client.closing = true
client.mutex.Unlock()
return client.codec.Close()
}
// Go invokes the function asynchronously. It returns the Call structure representing
// the invocation. The done channel will signal when the call is complete by returning
// the same Call object. If done is nil, Go will allocate a new channel.
// If non-nil, done must be buffered or Go will deliberately crash.
func (client *Client) Go(serviceMethod string, args interface{}, reply interface{}, done chan *Call, queueMode bool) *Call {
func (slf *Client) AsycGo(rpcHandler IRpcHandler,mutiCoroutine bool,serviceMethod string,callback reflect.Value, args interface{},replyParam interface{}) error {
call := new(Call)
call.ServiceMethod = serviceMethod
call.Args = args
call.Reply = reply
if done == nil {
done = make(chan *Call, 10) // buffered.
} else {
// If caller passes done != nil, it must arrange that
// done has enough buffer for the number of simultaneous
// RPCs that will be using that channel. If the channel
// is totally unbuffered, it's best not to run at all.
if cap(done) == 0 {
log.Panic("rpc: done channel is unbuffered")
}
call.Reply = replyParam
call.callback = &callback
call.rpcHandler = rpcHandler
request := &RpcRequest{}
request.NoReply = false
request.MutiCoroutine = mutiCoroutine
call.Arg = args
slf.pendingLock.Lock()
slf.startSeq += 1
call.Seq = slf.startSeq
request.Seq = slf.startSeq
slf.pending[call.Seq] = call
slf.pendingLock.Unlock()
request.ServiceMethod = serviceMethod
var herr error
request.InParam,herr = processor.Marshal(args)
if herr != nil {
return herr
}
call.Done = done
client.send(call, queueMode)
bytes,err := processor.Marshal(request)
if err != nil {
return err
}
if slf.conn == nil {
return fmt.Errorf("Rpc server is disconnect,call %s is fail!",serviceMethod)
}
err = slf.conn.WriteMsg(bytes)
if err != nil {
call.Err = err
}
return call.Err
}
func (slf *Client) Go(noReply bool,mutiCoroutine bool,serviceMethod string, args interface{},reply interface{}) *Call {
call := new(Call)
call.done = make(chan *Call,1)
call.Reply = reply
request := &RpcRequest{}
request.MutiCoroutine = mutiCoroutine
request.NoReply = noReply
call.Arg = args
slf.pendingLock.Lock()
slf.startSeq += 1
call.Seq = slf.startSeq
request.Seq = slf.startSeq
slf.pending[call.Seq] = call
slf.pendingLock.Unlock()
request.ServiceMethod = serviceMethod
var herr error
request.InParam,herr = processor.Marshal(args)
if herr != nil {
call.Err = herr
return call
}
bytes,err := processor.Marshal(request)
if err != nil {
call.Err = err
return call
}
err = slf.conn.WriteMsg(bytes)
if err != nil {
call.Err = err
}
return call
}
// Call invokes the named function, waits for it to complete, and returns its error status.
func (client *Client) Call(serviceMethod string, args interface{}, reply interface{}) error {
select {
case call := <-client.Go(serviceMethod, args, reply, make(chan *Call, 1), false).Done:
return call.Error
case <-time.After(30 * time.Second):
type RequestHandler func(Returns interface{},Err error)
type RpcRequest struct {
//packhead
Seq uint64 // sequence number chosen by client
ServiceMethod string // format: "Service.Method"
NoReply bool //是否需要返回
MutiCoroutine bool // 是否多协程模式
//packbody
InParam []byte
localReply interface{}
localParam interface{} //本地调用的参数列表
requestHandle RequestHandler
callback *reflect.Value
}
type RpcResponse struct {
//head
Seq uint64 // sequence number chosen by client
Err error
//returns
Returns []byte
}
func (slf *Client) Run(){
for {
bytes,err := slf.conn.ReadMsg()
if err != nil {
slf.Close()
slf.Start()
}
//1.解析head
respone := &RpcResponse{}
err = processor.Unmarshal(bytes,respone)
if err != nil {
log.Error("rpcClient Unmarshal head error,error:%+v",err)
continue
}
slf.pendingLock.Lock()
v,ok := slf.pending[respone.Seq]
if ok == false {
log.Error("rpcClient cannot find seq %d in pending",respone.Seq)
slf.pendingLock.Unlock()
}else {
delete(slf.pending,respone.Seq)
slf.pendingLock.Unlock()
err = processor.Unmarshal(respone.Returns,v.Reply)
if err != nil {
log.Error("rpcClient Unmarshal body error,error:%+v",err)
continue
}
if v.callback.IsValid() {
v.rpcHandler.(*RpcHandler).callResponeCallBack<-v
}else{
//发送至接受者
v.done <- v
}
}
}
//call := <-client.Go(serviceMethod, args, reply, make(chan *Call, 1)).Done
return errors.New(fmt.Sprintf("Call RPC %s is time out 30s", serviceMethod))
}
func (slf *Client) OnClose(){
if slf.blocalhost== false{
//关闭时，重新连接
slf.Start()
}
}

90
rpc/debug.go
View File

@@ -1,90 +0,0 @@
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package rpc
/*
Some HTML presented at http://machine:port/debug/rpc
Lists services, their methods, and some statistics, still rudimentary.
*/
import (
"fmt"
"html/template"
"net/http"
"sort"
)
const debugText = `<html>
<body>
<title>Services</title>
{{range .}}
<hr>
Service {{.Name}}
<hr>
<table>
<th align=center>Method</th><th align=center>Calls</th>
{{range .Method}}
<tr>
<td align=left font=fixed>{{.Name}}({{.Type.ArgType}}, {{.Type.ReplyType}}) error</td>
<td align=center>{{.Type.NumCalls}}</td>
</tr>
{{end}}
</table>
{{end}}
</body>
</html>`
var debug = template.Must(template.New("RPC debug").Parse(debugText))
// If set, print log statements for internal and I/O errors.
var debugLog = false
type debugMethod struct {
Type *methodType
Name string
}
type methodArray []debugMethod
type debugService struct {
Service *service
Name string
Method methodArray
}
type serviceArray []debugService
func (s serviceArray) Len() int { return len(s) }
func (s serviceArray) Less(i, j int) bool { return s[i].Name < s[j].Name }
func (s serviceArray) Swap(i, j int) { s[i], s[j] = s[j], s[i] }
func (m methodArray) Len() int { return len(m) }
func (m methodArray) Less(i, j int) bool { return m[i].Name < m[j].Name }
func (m methodArray) Swap(i, j int) { m[i], m[j] = m[j], m[i] }
type debugHTTP struct {
*Server
}
// Runs at /debug/rpc
func (server debugHTTP) ServeHTTP(w http.ResponseWriter, req *http.Request) {
// Build a sorted version of the data.
var services serviceArray
server.serviceMap.Range(func(snamei, svci interface{}) bool {
svc := svci.(*service)
ds := debugService{svc, snamei.(string), make(methodArray, 0, len(svc.method))}
for mname, method := range svc.method {
ds.Method = append(ds.Method, debugMethod{method, mname})
}
sort.Sort(ds.Method)
services = append(services, ds)
return true
})
sort.Sort(services)
err := debug.Execute(w, services)
if err != nil {
fmt.Fprintln(w, "rpc: error executing template:", err.Error())
}
}

1
rpc/gobrpc/processor.go Normal file
View File

@@ -0,0 +1 @@
package gobrpc

22
rpc/jsonprocessor.go Normal file
View File

@@ -0,0 +1,22 @@
package rpc
import (
"encoding/json"
)
type JsonProcessor struct {
}
func (slf *JsonProcessor) Marshal(v interface{}) ([]byte, error){
return json.Marshal(v)
}
func (slf *JsonProcessor) Unmarshal(data []byte, v interface{}) error{
return json.Unmarshal(data,v)
}

1
rpc/netrpc.go Normal file
View File

@@ -0,0 +1 @@
package rpc

377
rpc/rpchandler.go Normal file
View File

@@ -0,0 +1,377 @@
package rpc
import (
"fmt"
"github.com/duanhf2012/originnet/log"
"reflect"
"strings"
"unicode"
"unicode/utf8"
)
type FuncRpcClient func(serviceMethod string) ([]*Client,error)
type FuncRpcServer func() (*Server)
var NilError = reflect.Zero(reflect.TypeOf((*error)(nil)).Elem())
type RpcMethodInfo struct {
method reflect.Method
iparam interface{}
oParam reflect.Value
}
type RpcHandler struct {
callRequest chan *RpcRequest
rpcHandler IRpcHandler
mapfunctons map[string]RpcMethodInfo
funcRpcClient FuncRpcClient
funcRpcServer FuncRpcServer
callResponeCallBack chan *Call //异步返回的回调
}
type IRpcHandler interface {
GetName() string
InitRpcHandler(rpcHandler IRpcHandler,getClientFun FuncRpcClient,getServerFun FuncRpcServer)
GetRpcHandler() IRpcHandler
PushRequest(callinfo *RpcRequest)
HandlerRpcRequest(request *RpcRequest)
HandlerRpcResponeCB(call *Call)
GetRpcRequestChan() chan *RpcRequest
GetRpcResponeChan() chan *Call
CallMethod(ServiceMethod string,param interface{},reply interface{}) error
}
func (slf *RpcHandler) GetRpcHandler() IRpcHandler{
return slf.rpcHandler
}
func (slf *RpcHandler) InitRpcHandler(rpcHandler IRpcHandler,getClientFun FuncRpcClient,getServerFun FuncRpcServer) {
slf.callRequest = make(chan *RpcRequest,100000)
slf.callResponeCallBack = make(chan *Call,100000)
slf.rpcHandler = rpcHandler
slf.mapfunctons = map[string]RpcMethodInfo{}
slf.funcRpcClient = getClientFun
slf.funcRpcServer = getServerFun
slf.RegisterRpc(rpcHandler)
}
// Is this an exported - upper case - name?
func isExported(name string) bool {
rune, _ := utf8.DecodeRuneInString(name)
return unicode.IsUpper(rune)
}
// Is this type exported or a builtin?
func (slf *RpcHandler) isExportedOrBuiltinType(t reflect.Type) bool {
for t.Kind() == reflect.Ptr {
t = t.Elem()
}
// PkgPath will be non-empty even for an exported type,
// so we need to check the type name as well.
return isExported(t.Name()) || t.PkgPath() == ""
}
func (slf *RpcHandler) suitableMethods(method reflect.Method) error {
//只有RPC_开头的才能被调用
if strings.Index(method.Name,"RPC_")!=0 {
return nil
}
//取出输入参数类型
var rpcMethodInfo RpcMethodInfo
typ := method.Type
if typ.NumOut() != 1 {
return fmt.Errorf("%s The number of returned arguments must be 1!",method.Name)
}
if typ.Out(0).String() != "error" {
return fmt.Errorf("%s The return parameter must be of type error!",method.Name)
}
if typ.NumIn() != 3 {
return fmt.Errorf("%s The number of input arguments must be 1!",method.Name)
}
if slf.isExportedOrBuiltinType(typ.In(1)) == false || slf.isExportedOrBuiltinType(typ.In(2)) == false {
return fmt.Errorf("%s Unsupported parameter types!",method.Name)
}
rpcMethodInfo.iparam = reflect.New(typ.In(1).Elem()).Interface() //append(rpcMethodInfo.iparam,)
rpcMethodInfo.oParam = reflect.New(typ.In(2).Elem())
rpcMethodInfo.method = method
slf.mapfunctons[slf.rpcHandler.GetName()+"."+method.Name] = rpcMethodInfo
return nil
}
func (slf *RpcHandler) RegisterRpc(rpcHandler IRpcHandler) error {
typ := reflect.TypeOf(rpcHandler)
for m:=0;m<typ.NumMethod();m++{
method := typ.Method(m)
err := slf.suitableMethods(method)
if err != nil {
panic(err)
}
}
return nil
}
func (slf *RpcHandler) PushRequest(req *RpcRequest) {
slf.callRequest <- req
}
func (slf *RpcHandler) GetRpcRequestChan() (chan *RpcRequest) {
return slf.callRequest
}
func (slf *RpcHandler) GetRpcResponeChan() chan *Call{
return slf.callResponeCallBack
}
func (slf *RpcHandler) HandlerRpcResponeCB(call *Call){
if call.Err == nil {
call.callback.Call([]reflect.Value{reflect.ValueOf(call.Reply),NilError})
}else{
call.callback.Call([]reflect.Value{reflect.ValueOf(call.Reply),reflect.ValueOf(call.Err)})
}
}
func (slf *RpcHandler) HandlerRpcRequest(request *RpcRequest) {
v,ok := slf.mapfunctons[request.ServiceMethod]
if ok == false {
err := fmt.Errorf("RpcHandler %s cannot find %s",slf.rpcHandler.GetName(),request.ServiceMethod)
log.Error("%s",err.Error())
if request.requestHandle!=nil {
request.requestHandle(nil,err)
}
return
}
var paramList []reflect.Value
var err error
if request.localParam==nil{
err = processor.Unmarshal(request.InParam,&v.iparam)
if err!=nil {
rerr := fmt.Errorf("Call Rpc %s Param error %+v",request.ServiceMethod,err)
log.Error("%s",rerr.Error())
if request.requestHandle!=nil {
request.requestHandle(nil, rerr)
}
}
}else {
v.iparam = request.localParam
}
paramList = append(paramList,reflect.ValueOf(slf.GetRpcHandler())) //接受者
if request.localReply!=nil {
v.oParam = reflect.ValueOf(request.localReply)
}
paramList = append(paramList,reflect.ValueOf(v.iparam))
paramList = append(paramList,v.oParam) //输出参数
returnValues := v.method.Func.Call(paramList)
errInter := returnValues[0].Interface()
if errInter != nil {
err = errInter.(error)
}
if request.requestHandle!=nil {
request.requestHandle(v.oParam.Interface(), err)
}
}
func (slf *RpcHandler) CallMethod(ServiceMethod string,param interface{},reply interface{}) error{
var err error
v,ok := slf.mapfunctons[ServiceMethod]
if ok == false {
err = fmt.Errorf("RpcHandler %s cannot find %s",slf.rpcHandler.GetName(),ServiceMethod)
log.Error("%s",err.Error())
return err
}
var paramList []reflect.Value
paramList = append(paramList,reflect.ValueOf(slf.GetRpcHandler())) //接受者
paramList = append(paramList,reflect.ValueOf(param))
paramList = append(paramList,reflect.ValueOf(reply)) //输出参数
returnValues := v.method.Func.Call(paramList)
errInter := returnValues[0].Interface()
if errInter != nil {
err = errInter.(error)
}
return err
}
func (slf *RpcHandler) goRpc(serviceMethod string,mutiCoroutine bool,args interface{}) error {
pClientList,err := slf.funcRpcClient(serviceMethod)
if err != nil {
log.Error("Call serviceMethod is error:%+v!",err)
return err
}
if len(pClientList) > 1 {
log.Error("Cannot call more then 1 node!")
return fmt.Errorf("Cannot call more then 1 node!")
}
//2.rpcclient调用
//如果调用本结点服务
pClient := pClientList[0]
if pClient.blocalhost == true {
pLocalRpcServer:=slf.funcRpcServer()
//判断是否是同一服务
sMethod := strings.Split(serviceMethod,".")
if len(sMethod)!=2 {
err := fmt.Errorf("Call serviceMethod %s is error!",serviceMethod)
log.Error("%+v",err)
return err
}
//调用自己rpcHandler处理器
if sMethod[0] == slf.rpcHandler.GetName() { //自己服务调用
//
return pLocalRpcServer.myselfRpcHandlerGo(sMethod[0],sMethod[1],args,nil)
}
//其他的rpcHandler的处理器
pCall := pLocalRpcServer.rpcHandlerGo(true,mutiCoroutine,sMethod[0],sMethod[1],args,nil)
return pCall.Err
}
//跨node调用
pCall := pClient.Go(true,mutiCoroutine,serviceMethod,args,nil)
return pCall.Err
}
func (slf *RpcHandler) callRpc(serviceMethod string,mutiCoroutine bool,args interface{},reply interface{}) error {
pClientList,err := slf.funcRpcClient(serviceMethod)
if err != nil {
log.Error("Call serviceMethod is error:%+v!",err)
return err
}
if len(pClientList) > 1 {
log.Error("Cannot call more then 1 node!")
return fmt.Errorf("Cannot call more then 1 node!")
}
//2.rpcclient调用
//如果调用本结点服务
pClient := pClientList[0]
if pClient.blocalhost == true {
pLocalRpcServer:=slf.funcRpcServer()
//判断是否是同一服务
sMethod := strings.Split(serviceMethod,".")
if len(sMethod)!=2 {
err := fmt.Errorf("Call serviceMethod %s is error!",serviceMethod)
log.Error("%+v",err)
return err
}
//调用自己rpcHandler处理器
if sMethod[0] == slf.rpcHandler.GetName() { //自己服务调用
//
return pLocalRpcServer.myselfRpcHandlerGo(sMethod[0],sMethod[1],args,reply)
}
//其他的rpcHandler的处理器
pCall := pLocalRpcServer.rpcHandlerGo(false,mutiCoroutine,sMethod[0],sMethod[1],args,reply)
pResult := pCall.Done()
return pResult.Err
}
//跨node调用
pCall := pClient.Go(false,mutiCoroutine,serviceMethod,args,reply)
pResult := pCall.Done()
return pResult.Err
}
func (slf *RpcHandler) asyncCallRpc(serviceMethod string,mutiCoroutine bool,args interface{},callback interface{}) error {
fVal := reflect.ValueOf(callback)
if fVal.Kind()!=reflect.Func{
return fmt.Errorf("input function is error!")
}
reply := reflect.New(fVal.Type().In(0).Elem()).Interface()
pClientList,err := slf.funcRpcClient(serviceMethod)
if err != nil {
log.Error("Call serviceMethod is error:%+v!",err)
return err
}
if len(pClientList) > 1 {
log.Error("Cannot call more then 1 node!")
return fmt.Errorf("Cannot call more then 1 node!")
}
//2.rpcclient调用
//如果调用本结点服务
pClient := pClientList[0]
if pClient.blocalhost == true {
pLocalRpcServer:=slf.funcRpcServer()
//判断是否是同一服务
sMethod := strings.Split(serviceMethod,".")
if len(sMethod)!=2 {
err := fmt.Errorf("Call serviceMethod %s is error!",serviceMethod)
log.Error("%+v",err)
return err
}
//调用自己rpcHandler处理器
if sMethod[0] == slf.rpcHandler.GetName() { //自己服务调用
err := pLocalRpcServer.myselfRpcHandlerGo(sMethod[0],sMethod[1],args,reply)
if err == nil {
fVal.Call([]reflect.Value{reflect.ValueOf(reply),NilError})
}else{
fVal.Call([]reflect.Value{reflect.ValueOf(reply),reflect.ValueOf(err)})
}
}
//其他的rpcHandler的处理器
if callback!=nil {
return pLocalRpcServer.rpcHandlerAsyncGo(slf,false,mutiCoroutine,sMethod[0],sMethod[1],args,reply,fVal)
}
pCall := pLocalRpcServer.rpcHandlerGo(false,mutiCoroutine,sMethod[0],sMethod[1],args,reply)
pResult := pCall.Done()
return pResult.Err
}
//跨node调用
return pClient.AsycGo(slf,mutiCoroutine,serviceMethod,fVal,args,reply)
}
func (slf *RpcHandler) GetName() string{
return slf.rpcHandler.GetName()
}
//func (slf *RpcHandler) asyncCallRpc(serviceMethod string,mutiCoroutine bool,callback interface{},args ...interface{}) error {
//func (slf *RpcHandler) callRpc(serviceMethod string,reply interface{},mutiCoroutine bool,args ...interface{}) error {
//func (slf *RpcHandler) goRpc(serviceMethod string,mutiCoroutine bool,args ...interface{}) error {
//(reply *int,err error) {}
func (slf *RpcHandler) AsyncCall(serviceMethod string,args interface{},callback interface{}) error {
return slf.asyncCallRpc(serviceMethod,false,args,callback)
}
func (slf *RpcHandler) GRAsyncCall(serviceMethod string,args interface{},callback interface{}) error {
return slf.asyncCallRpc(serviceMethod,true,args,callback)
}
func (slf *RpcHandler) Call(serviceMethod string,args interface{},reply interface{}) error {
return slf.callRpc(serviceMethod,false,args,reply)
}
func (slf *RpcHandler) GRCall(serviceMethod string,args interface{},reply interface{}) error {
return slf.callRpc(serviceMethod,true,args,reply)
}
func (slf *RpcHandler) Go(serviceMethod string,args interface{}) error {
return slf.goRpc(serviceMethod,false,args)
}
func (slf *RpcHandler) GRGo(serviceMethod string,args interface{}) error {
return slf.goRpc(serviceMethod,true,args)
}

1015
rpc/server.go
View File

File diff suppressed because it is too large Load Diff

78
service/DeadForMonitor.go
View File

@@ -1,78 +0,0 @@
package service
import (
"fmt"
"github.com/duanhf2012/origin/util"
"runtime/pprof"
"time"
)
type ModuleMontior struct {
mapModule *util.MapEx
}
type ModuleInfo struct {
enterStartTm int64
mNameInfo string
}
var moduleMontior ModuleMontior
func MonitorEnter(uuid string,strMonitorInfo string){
if moduleMontior.mapModule == nil {
return
}
moduleMontior.mapModule.Set(uuid, &ModuleInfo{enterStartTm:time.Now().Unix(),mNameInfo:strMonitorInfo})
}
func MonitorLeave(uuid string){
if moduleMontior.mapModule == nil {
return
}
moduleMontior.mapModule.Del(uuid)
}
func ReportDeadFor(){
if moduleMontior.mapModule == nil {
return
}
moduleMontior.mapModule.RLockRange(func(key interface{}, value interface{}) {
if value != nil {
pModuleInfo := value.(*ModuleInfo)
//超过5分钟认为dead for
if time.Now().Unix() - pModuleInfo.enterStartTm > 300 {
GetLogger().Printf(LEVER_FATAL, "module is %s, Dead cycle\n", pModuleInfo.mNameInfo)
}
}
})
}
func EnableDeadForMonitor(checkInterval time.Duration){
moduleMontior.mapModule = util.NewMapEx()
var tmInval util.Timer
tmInval.SetupTimer(int32(checkInterval.Milliseconds()))
go func(){
for {
time.Sleep(time.Second*5)
if tmInval.CheckTimeOut(){
ReportDeadFor()
ReportPprof()
}
}
}()
}
func ReportPprof(){
strReport := ""
for _, p := range pprof.Profiles() {
strReport += fmt.Sprintf("Name %s,count %d\n",p.Name(),p.Count())
}
GetLogger().Printf(LEVER_INFO, "PProf %s\n", strReport)
}

37
service/Logger.go
View File

@@ -1,37 +0,0 @@
package service
import (
"fmt"
)
const (
LEVER_UNKNOW = 0
LEVER_DEBUG = 1
LEVER_INFO = 2
LEVER_WARN = 3
LEVER_ERROR = 4
LEVER_FATAL = 5
LEVEL_MAX = 6
)
var defaultLogger = &LoggerFmt{}
type ILogger interface {
Printf(level uint, format string, v ...interface{})
Print(level uint, v ...interface{})
SetLogLevel(level uint)
}
type LoggerFmt struct {
}
func (slf *LoggerFmt) Printf(level uint, format string, v ...interface{}) {
fmt.Printf(format, v...)
fmt.Println("")
}
func (slf *LoggerFmt) Print(level uint, v ...interface{}) {
fmt.Println(v...)
}
func (slf *LoggerFmt) SetLogLevel(level uint) {
//do nothing
}

442
service/Module.go
View File

@@ -2,361 +2,191 @@ package service
import (
"fmt"
"github.com/duanhf2012/origin/util"
"github.com/duanhf2012/origin/util/uuid"
"runtime/debug"
"sync"
"sync/atomic"
"github.com/duanhf2012/originnet/event"
"github.com/duanhf2012/originnet/util/timer"
"time"
)
const (
//ModuleNone ...
INIT_AUTO_INCREMENT = 0
)
const InitModuleId = 1e18
type IModule interface {
OnInit() error //Module初始化时调用
OnRun() bool //Module运行时调用
OnEndRun()
SetModuleId(moduleId uint32) //手动设置ModuleId
GetModuleId() uint32 //获取ModuleId
GetModuleById(moduleId uint32) IModule //通过ModuleId获取Module
AddModule(module IModule) uint32 //添加Module
ReleaseModule(moduleId uint32) bool //释放Module
SetSelf(module IModule) //设置保存自己interface
GetSelf() IModule //获取自己interface
SetOwner(module IModule) //设置父Module
GetOwner() IModule //获取父Module
SetOwnerService(iservice IService) //设置拥有者服务
GetOwnerService() IService //获取拥有者服务
GetRoot() IModule //获取Root根Module
RunModule(module IModule) //手动运行Module
InitModule(exit chan bool, pwaitGroup *sync.WaitGroup) error //手动初始化Module
getBaseModule() *BaseModule //获取BaseModule指针
IsInit() bool
SetUnOnRun()
getUnOnRun() bool
SetModuleId(moduleId int64) bool
GetModuleId() int64
AddModule(module IModule) (int64,error)
GetModule(moduleId int64) IModule
GetAncestor()IModule
ReleaseModule(moduleId int64)
NewModuleId() int64
GetParent()IModule
OnInit() error
OnRelease()
getBaseModule() IModule
GetService() IService
GetEventChan() chan *event.Event
}
type BaseModule struct {
moduleId uint32
ownerService IService
mapModule map[uint32]IModule
ownerModule IModule
selfModule IModule
CurrMaxModuleId uint32
corouterstatus int32 //0表示运行状态 //1释放消亡状态
//1.管理各模块树层关系
//2.提供定时器常用工具
type Module struct {
moduleId int64
parent IModule //父亲
self IModule //父亲
child map[int64]IModule //孩子们
mapActiveTimer map[*timer.Timer]interface{}
mapActiveCron map[*timer.Cron]interface{}
moduleLocker sync.RWMutex
ExitChan chan bool
WaitGroup *sync.WaitGroup
bInit bool
dispatcher *timer.Dispatcher //timer
recoverCount int8
bUnOnRun bool
//根结点
ancestor IModule //始祖
seedModuleId int64 //模块id种子
descendants map[int64]IModule//始祖的后裔们
//事件管道
event.EventProcessor
//eventChan chan *SEvent
}
func (slf *BaseModule) GetRoot() IModule {
currentOwner := slf.GetSelf()
for {
owner := currentOwner.GetOwner()
if owner == currentOwner {
return owner
}
currentOwner = owner
}
}
func (slf *BaseModule) SetModuleId(moduleId uint32) {
slf.moduleId = moduleId
}
func (slf *BaseModule) GetModuleId() uint32 {
return slf.moduleId
}
func (slf *BaseModule) GetModuleById(moduleId uint32) IModule {
slf.moduleLocker.RLock()
ret, ok := slf.mapModule[moduleId]
slf.moduleLocker.RUnlock()
if ok == false {
return nil
}
return ret
}
func (slf *BaseModule) GetModuleCount() int {
slf.moduleLocker.RLock()
moduleCount := len(slf.mapModule)
slf.moduleLocker.RUnlock()
return moduleCount
}
func (slf *BaseModule) genModuleId() uint32 {
slf.CurrMaxModuleId++
return slf.CurrMaxModuleId
}
func (slf *BaseModule) deleteModule(baseModule *BaseModule) bool {
for _, subModule := range baseModule.mapModule {
baseModule.deleteModule(subModule.getBaseModule())
}
atomic.AddInt32(&baseModule.corouterstatus, 1)
//fmt.Printf("Delete %T->%T\n", slf.GetSelf(), baseModule.GetSelf())
baseModule.ownerService = nil
baseModule.mapModule = nil
baseModule.ownerModule = nil
baseModule.selfModule = nil
delete(slf.mapModule, baseModule.GetModuleId())
baseModule.moduleLocker.Unlock()
return true
}
func (slf *BaseModule) LockTree(rootModule *BaseModule) {
rootModule.moduleLocker.Lock()
//fmt.Printf("Lock %T\n", rootModule.GetSelf())
for _, pModule := range rootModule.mapModule {
slf.LockTree(pModule.getBaseModule())
//pModule.getBaseModule().moduleLocker.Lock()
}
}
func (slf *BaseModule) UnLockTree(rootModule *BaseModule) {
for _, pModule := range rootModule.mapModule {
pModule.getBaseModule().moduleLocker.Unlock()
}
rootModule.moduleLocker.Unlock()
}
func (slf *BaseModule) ReleaseModule(moduleId uint32) bool {
slf.moduleLocker.Lock()
module, ok := slf.mapModule[moduleId]
if ok == false {
slf.moduleLocker.Unlock()
GetLogger().Printf(LEVER_FATAL, "RemoveModule fail %d...", moduleId)
func (slf *Module) SetModuleId(moduleId int64) bool{
if moduleId > 0 {
return false
}
//锁住被结点树
slf.LockTree(module.getBaseModule())
slf.deleteModule(module.getBaseModule())
delete(slf.mapModule, moduleId)
//fmt.Printf("++Release %T -- %+v\n", slf.GetSelf(), slf.mapModule)
slf.moduleLocker.Unlock()
slf.moduleId = moduleId
return true
}
func (slf *BaseModule) IsRoot() bool {
return slf.GetOwner() == slf.GetSelf()
func (slf *Module) GetModuleId() int64{
return slf.moduleId
}
func (slf *BaseModule) GetSelf() IModule {
if slf.selfModule == nil {
return slf
}
return slf.selfModule
func (slf *Module) OnInit() error{
return nil
}
func (slf *BaseModule) SetUnOnRun() {
slf.bUnOnRun = true
}
func (slf *BaseModule) getUnOnRun() bool {
return slf.bUnOnRun
}
func (slf *BaseModule) AddModule(module IModule) uint32 {
//消亡状态不允许加入模块
if atomic.LoadInt32(&slf.corouterstatus) != 0 {
GetLogger().Printf(LEVER_ERROR, "%T Cannot AddModule %T", slf.GetSelf(), module.GetSelf())
return 0
func (slf *Module) AddModule(module IModule) (int64,error){
pAddModule := module.getBaseModule().(*Module)
if pAddModule.GetModuleId()==0 {
pAddModule.moduleId = slf.NewModuleId()
}
pModule := slf.GetModuleById(module.GetModuleId())
if pModule != nil {
GetLogger().Printf(LEVER_ERROR, "%T Cannot AddModule %T,moduleid %d is repeat!", slf.GetSelf(), module.GetSelf(), module.GetModuleId())
return 0
if slf.child == nil {
slf.child = map[int64]IModule{}
}
//如果没有设置，自动生成ModuleId
slf.moduleLocker.Lock()
var genid uint32
if module.GetModuleId() == 0 {
genid = slf.genModuleId()
module.SetModuleId(genid)
}
module.getBaseModule().selfModule = module
if slf.GetOwner() != nil {
if slf.IsRoot() {
//root owner为自己
module.SetOwner(slf.GetOwner())
} else {
module.SetOwner(slf.GetSelf())
}
}
//设置模块退出信号捕获
module.InitModule(slf.ExitChan, slf.WaitGroup)
//存入父模块中
if slf.mapModule == nil {
slf.mapModule = make(map[uint32]IModule)
}
_, ok := slf.mapModule[module.GetModuleId()]
_,ok := slf.child[module.GetModuleId()]
if ok == true {
slf.moduleLocker.Unlock()
GetLogger().Printf(LEVER_ERROR, "check mapModule %#v id is %d ,%d is fail...", module, module.GetModuleId(), genid)
return 0
return 0,fmt.Errorf("Exists module id %d",module.GetModuleId())
}
slf.mapModule[module.GetModuleId()] = module
pAddModule.self = module
pAddModule.parent = slf.self
pAddModule.dispatcher = slf.GetAncestor().getBaseModule().(*Module).dispatcher
pAddModule.ancestor = slf.ancestor
slf.moduleLocker.Unlock()
//运行模块
GetLogger().Printf(LEVER_INFO, "Start Init module %T.", module)
err := module.OnInit()
if err != nil {
delete(slf.mapModule, module.GetModuleId())
GetLogger().Printf(LEVER_ERROR, "End Init module %T id is %d is fail,reason:%v...", module, module.GetModuleId(), err)
return 0
}
initErr := module.getBaseModule().OnInit()
if initErr != nil {
delete(slf.mapModule, module.GetModuleId())
GetLogger().Printf(LEVER_ERROR, "OnInit module %T id is %d is fail,reason:%v...", module, module.GetModuleId(), initErr)
return 0
return 0,err
}
GetLogger().Printf(LEVER_INFO, "End Init module %T.", module)
if module.getUnOnRun() == false {
go module.RunModule(module)
slf.child[module.GetModuleId()] = module
slf.ancestor.getBaseModule().(*Module).descendants[module.GetModuleId()] = module
return module.GetModuleId(),nil
}
func (slf *Module) ReleaseModule(moduleId int64){
//pBaseModule := slf.GetModule(moduleId).getBaseModule().(*Module)
pModule := slf.GetModule(moduleId).getBaseModule().(*Module)
//释放子孙
for id,_ := range pModule.child {
slf.ReleaseModule(id)
}
pModule.self.OnRelease()
for pTimer,_ := range pModule.mapActiveTimer {
pTimer.Stop()
}
return module.GetModuleId()
}
func (slf *BaseModule) OnInit() error {
slf.bInit = true
return nil
}
func (slf *BaseModule) OnRun() bool {
return false
}
func (slf *BaseModule) OnEndRun() {
}
func (slf *BaseModule) SetOwner(ownerModule IModule) {
slf.ownerModule = ownerModule
}
func (slf *BaseModule) SetSelf(module IModule) {
slf.selfModule = module
}
func (slf *BaseModule) GetOwner() IModule {
if slf.ownerModule == nil {
return slf
for pCron,_ := range pModule.mapActiveCron {
pCron.Stop()
}
return slf.ownerModule
delete(slf.child,moduleId)
delete (slf.ancestor.getBaseModule().(*Module).descendants,moduleId)
//清理被删除的Module
pModule.self = nil
pModule.parent = nil
pModule.child = nil
pModule.mapActiveTimer = nil
pModule.mapActiveCron = nil
pModule.dispatcher = nil
pModule.ancestor = nil
pModule.descendants = nil
}
func (slf *BaseModule) GetOwnerService() IService {
return slf.ownerService
func (slf *Module) NewModuleId() int64{
slf.ancestor.getBaseModule().(*Module).seedModuleId+=1
return slf.ancestor.getBaseModule().(*Module).seedModuleId
}
func (slf *BaseModule) SetOwnerService(iservice IService) {
slf.ownerService = iservice
func (slf *Module) GetAncestor()IModule{
return slf.ancestor
}
func (slf *BaseModule) InitModule(exit chan bool, pwaitGroup *sync.WaitGroup) error {
slf.CurrMaxModuleId = INIT_AUTO_INCREMENT
slf.WaitGroup = pwaitGroup
slf.ExitChan = exit
return nil
func (slf *Module) GetModule(moduleId int64) IModule{
iModule,ok := slf.GetAncestor().getBaseModule().(*Module).descendants[moduleId]
if ok == false{
return nil
}
return iModule
}
func (slf *BaseModule) getBaseModule() *BaseModule {
func (slf *Module) getBaseModule() IModule{
return slf
}
func (slf *BaseModule) IsInit() bool {
return slf.bInit
func (slf *Module) GetParent()IModule{
return slf.parent
}
func (slf *BaseModule) RunModule(module IModule) {
GetLogger().Printf(LEVER_INFO, "Start Run module %T ...", module)
defer func() {
if r := recover(); r != nil {
var coreInfo string
coreInfo = string(debug.Stack())
coreInfo += "\n" + fmt.Sprintf("Core module is %T, try count %d. core information is %v\n", module, slf.recoverCount, r)
GetLogger().Printf(LEVER_FATAL, coreInfo)
slf.recoverCount += 1
//重试3次
if slf.recoverCount < 10 {
go slf.RunModule(slf.GetSelf())
} else {
GetLogger().Printf(LEVER_FATAL, "Routine %T.OnRun has exited!", module)
}
}
}()
//运行所有子模块
timer := util.Timer{}
timer.SetupTimer(1000)
slf.WaitGroup.Add(1)
defer slf.WaitGroup.Done()
uuidkey := uuid.Rand().HexEx()
moduleTypeName := fmt.Sprintf("%T",module)
for {
if atomic.LoadInt32(&slf.corouterstatus) != 0 {
module.OnEndRun()
GetLogger().Printf(LEVER_INFO, "OnEndRun module %T ...", module)
break
}
//每500ms检查退出
if timer.CheckTimeOut() {
select {
case <-slf.ExitChan:
module.OnEndRun()
GetLogger().Printf(LEVER_INFO, "OnEndRun module %T...", module)
return
default:
}
}
MonitorEnter(uuidkey,moduleTypeName)
if module.OnRun() == false {
module.OnEndRun()
MonitorLeave(uuidkey)
GetLogger().Printf(LEVER_INFO, "OnEndRun module %T...", module)
return
}
MonitorLeave(uuidkey)
func (slf *Module) AfterFunc(d time.Duration, cb func()) *timer.Timer {
if slf.mapActiveTimer == nil {
slf.mapActiveTimer =map[*timer.Timer]interface{}{}
}
tm := slf.dispatcher.AfterFuncEx(d,func(t *timer.Timer){
cb()
delete(slf.mapActiveTimer,t)
})
slf.mapActiveTimer[tm] = nil
return tm
}
func (slf *Module) CronFunc(cronExpr *timer.CronExpr, cb func()) *timer.Cron {
if slf.mapActiveCron == nil {
slf.mapActiveCron =map[*timer.Cron]interface{}{}
}
cron := slf.dispatcher.CronFuncEx(cronExpr, func(cron *timer.Cron) {
cb()
delete(slf.mapActiveCron,cron)
})
slf.mapActiveCron[cron] = nil
return cron
}
func (slf *Module) OnRelease(){
}
func (slf *Module) GetService() IService {
return slf.GetAncestor().(IService)
}

153
service/Service.go
View File

@@ -1,85 +1,126 @@
package service
import (
"fmt"
"github.com/duanhf2012/originnet/rpc"
"github.com/duanhf2012/originnet/util/timer"
"reflect"
"strings"
"sync"
"sync/atomic"
)
type MethodInfo struct {
Fun reflect.Value
ParamList []reflect.Value
types reflect.Type
}
var closeSig chan bool
var timerDispatcherLen = 10
type IService interface {
Init(Iservice IService) error
Init(iservice IService,getClientFun rpc.FuncRpcClient,getServerFun rpc.FuncRpcServer,serviceCfg interface{})
GetName() string
OnInit() error
OnRun() bool
OnFetchService(iservice IService) error
OnSetupService(iservice IService) //其他服务被安装
OnRemoveService(iservice IService) //其他服务被安装
GetServiceName() string
SetServiceName(serviceName string) bool
GetServiceId() int
GetStatus() int
IsInit() bool
OnRelease()
Wait()
Start()
GetRpcHandler() rpc.IRpcHandler
GetServiceCfg()interface{}
}
type BaseService struct {
BaseModule
serviceid int
servicename string
Status int
type Service struct {
Module
rpc.RpcHandler //rpc
name string //service name
closeSig chan bool
wg sync.WaitGroup
this IService
serviceCfg interface{}
gorouterNum int32
startStatus bool
}
func (slf *BaseService) GetServiceId() int {
return slf.serviceid
func (slf *Service) Init(iservice IService,getClientFun rpc.FuncRpcClient,getServerFun rpc.FuncRpcServer,serviceCfg interface{}) {
slf.name = reflect.Indirect(reflect.ValueOf(iservice)).Type().Name()
slf.dispatcher =timer.NewDispatcher(timerDispatcherLen)
slf.this = iservice
slf.InitRpcHandler(iservice.(rpc.IRpcHandler),getClientFun,getServerFun)
//初始化祖先
slf.ancestor = iservice.(IModule)
slf.seedModuleId =InitModuleId
slf.descendants = map[int64]IModule{}
slf.serviceCfg = serviceCfg
slf.gorouterNum = 1
slf.this.OnInit()
}
func (slf *BaseService) GetServiceName() string {
return slf.servicename
}
func (slf *Service) SetGoRouterNum(gorouterNum int32) bool {
//已经开始状态不允许修改协程数量
if slf.startStatus == true {
return false
}
func (slf *BaseService) SetServiceName(serviceName string) bool {
slf.servicename = serviceName
slf.gorouterNum = gorouterNum
return true
}
func (slf *BaseService) GetStatus() int {
return slf.Status
func (slf *Service) Start() {
slf.startStatus = true
for i:=int32(0);i<slf.gorouterNum;i++{
slf.wg.Add(1)
go func(){
slf.Run()
}()
}
}
func (slf *BaseService) OnFetchService(iservice IService) error {
return nil
}
func (slf *BaseService) OnSetupService(iservice IService) {
}
func (slf *BaseService) OnRemoveService(iservice IService) {
}
func (slf *BaseService) Init(iservice IService) error {
slf.ownerService = iservice
if iservice.GetServiceName() == "" {
slf.servicename = fmt.Sprintf("%T", iservice)
parts := strings.Split(slf.servicename, ".")
if len(parts) != 2 {
GetLogger().Printf(LEVER_ERROR, "BaseService.Init: service name is error: %q", slf.servicename)
err := fmt.Errorf("BaseService.Init: service name is error: %q", slf.servicename)
return err
func (slf *Service) Run() {
defer slf.wg.Done()
var bStop = false
for{
rpcRequestChan := slf.GetRpcRequestChan()
rpcResponeCallBack := slf.GetRpcResponeChan()
eventChan := slf.GetEventChan()
select {
case <- closeSig:
bStop = true
case rpcRequest :=<- rpcRequestChan:
slf.GetRpcHandler().HandlerRpcRequest(rpcRequest)
case rpcResponeCB := <- rpcResponeCallBack:
slf.GetRpcHandler().HandlerRpcResponeCB(rpcResponeCB)
case event := <- eventChan:
slf.OnEventHandler(event)
case t := <- slf.dispatcher.ChanTimer:
t.Cb()
}
slf.servicename = parts[1]
if bStop == true {
if atomic.AddInt32(&slf.gorouterNum,-1)<=0 {
slf.startStatus = false
slf.OnRelease()
}
break
}
}
slf.serviceid = InstanceServiceMgr().GenServiceID()
}
func (slf *Service) GetName() string{
return slf.name
}
func (slf *Service) OnRelease(){
}
func (slf *Service) OnInit() error {
return nil
}
func (slf *Service) Wait(){
slf.wg.Wait()
}
func (slf *Service) GetServiceCfg()interface{}{
return slf.serviceCfg
}

132
service/servicemanager.go
View File

@@ -1,132 +0,0 @@
package service
import (
"os"
"sync"
)
type IServiceManager interface {
Setup(s IService) bool
Init(logger ILogger) bool
Start() bool
CreateServiceID() int
}
type CServiceManager struct {
genserviceid int
localserviceMap map[string]IService
logger ILogger
orderLocalService []string
}
func (slf *CServiceManager) Setup(s IService) bool {
s.(IModule).SetOwnerService(s)
s.(IModule).SetOwner(s.(IModule))
s.(IModule).SetSelf(s.(IModule))
slf.localserviceMap[s.GetServiceName()] = s
slf.orderLocalService = append(slf.orderLocalService, s.GetServiceName())
//通知其他服务已经安装
for _, is := range slf.localserviceMap {
//
is.OnSetupService(s)
s.OnSetupService(is)
}
return true
}
func (slf *CServiceManager) FindService(serviceName string) IService {
service, ok := slf.localserviceMap[serviceName]
if ok {
return service
}
return nil
}
type FetchService func(s IService) error
func (slf *CServiceManager) FetchService(s FetchService) IService {
for _, se := range slf.localserviceMap {
s(se)
}
return nil
}
func (slf *CServiceManager) Init(logger ILogger, exit chan bool, pwaitGroup *sync.WaitGroup) bool {
slf.logger = logger
for _, s := range slf.localserviceMap {
err := (s.(IModule)).InitModule(exit, pwaitGroup)
if err != nil {
slf.logger.Print(LEVER_FATAL, err)
return false
}
}
return true
}
func (slf *CServiceManager) Start() bool {
for _, sname := range slf.orderLocalService {
s := slf.FindService(sname)
GetLogger().Printf(LEVER_INFO, "Start Init module %T.", s.(IModule))
err := s.(IModule).OnInit()
if err != nil {
GetLogger().Printf(LEVER_ERROR, "Init module %T id is %d is fail,reason:%v...", s.(IModule), s.(IModule).GetModuleId(), err)
os.Exit(-1)
}
s.(IModule).getBaseModule().OnInit()
GetLogger().Printf(LEVER_INFO, "End Init module %T.", s.(IModule))
}
for _, sname := range slf.orderLocalService {
s := slf.FindService(sname)
go (s.(IModule)).RunModule(s.(IModule))
}
return true
}
func (slf *CServiceManager) GenServiceID() int {
slf.genserviceid += 1
return slf.genserviceid
}
func (slf *CServiceManager) GetLogger() ILogger {
ret := slf.logger
if ret == nil {
ret = defaultLogger
}
return ret
}
var self *CServiceManager
func InstanceServiceMgr() *CServiceManager {
if self == nil {
self = new(CServiceManager)
self.localserviceMap = make(map[string]IService)
return self
}
return self
}
func GetLogger() ILogger {
return InstanceServiceMgr().GetLogger()
}
func (slf *CServiceManager) IsFinishInit() bool {
for _, val := range slf.localserviceMap {
if val.IsInit() == false {
return false
}
}
return true
}

53
service/servicemgr.go Normal file
View File

@@ -0,0 +1,53 @@
package service
//本地所有的service
var mapServiceName map[string]IService
func init(){
mapServiceName = map[string]IService{}
}
func Init(chanCloseSig chan bool) {
closeSig=chanCloseSig
for _,s := range mapServiceName {
s.OnInit()
}
}
func Setup(s IService) bool {
_,ok := mapServiceName[s.GetName()]
if ok == true {
return false
}
//s.Init(s)
mapServiceName[s.GetName()] = s
return true
}
func GetService(servicename string) IService {
s,ok := mapServiceName[servicename]
if ok == false {
return nil
}
return s
}
func Start(){
for _,s := range mapServiceName {
s.Start()
}
}
func WaitStop(){
for _,s := range mapServiceName {
s.Wait()
}
}

37
servicelist/servicelist.go
View File

@@ -1,37 +0,0 @@
package servicelist
import (
"github.com/duanhf2012/origin/originnode"
"github.com/duanhf2012/origin/service"
)
var node = func() *originnode.COriginNode {
//1.新建OrginNode结点
node := originnode.NewOriginNode()
if node == nil {
println("originnode.NewOriginNode fail")
return nil
}
return node
}()
var serviceList []service.IService
// 增加服务列表 在init中调用
// 因为是init的时候调用 所以不用锁
func PushService(s service.IService) {
serviceList = append(serviceList, s)
}
//在main中调用该函数即可加载所有service
//debugCheckUrl "localhost:6060"
func Start(debugCheckUrl string) {
node.OpenDebugCheck(debugCheckUrl)
node.SetupService(serviceList...)
//5.初始化结点
node.Init()
//6.开始结点
node.Start()
}

237
sysmodule/DBModule.go
View File

@@ -4,13 +4,16 @@ import (
"database/sql"
"errors"
"fmt"
"github.com/duanhf2012/originnet/log"
"math/rand"
"net/url"
"reflect"
"strconv"
"strings"
"sync"
"time"
"github.com/duanhf2012/origin/service"
"github.com/duanhf2012/originnet/service"
_ "github.com/go-sql-driver/mysql"
)
@@ -20,9 +23,19 @@ const (
MAX_EXECUTE_FUN = 10000
)
type DBExecute struct {
syncExecuteFun chan SyncFun
syncExecuteExit chan bool
}
type PingExecute struct {
tickerPing *time.Ticker
pintExit chan bool
}
// DBModule ...
type DBModule struct {
service.BaseModule
service.Module
db *sql.DB
url string
username string
@@ -30,8 +43,12 @@ type DBModule struct {
dbname string
maxconn int
PrintTime time.Duration
syncExecuteFun chan SyncFun
pingCoroutine PingExecute
syncCoroutineNum int
executeList []DBExecute
waitGroup sync.WaitGroup
}
// Tx ...
@@ -72,43 +89,97 @@ type SyncDBResult struct {
sres chan DBResult
}
type SyncDBResultExCallBack func(dataList *DataSetList, err error)
type SyncQueryDBResultEx struct {
sres chan *DataSetList
err chan error
}
func (slf *SyncQueryDBResultEx) Get(timeoutMs int) (*DataSetList, error) {
timerC := time.NewTicker(time.Millisecond * time.Duration(timeoutMs)).C
select {
case <-timerC:
break
case err := <-slf.err:
dataset := <-slf.sres
return dataset, err
}
return nil, fmt.Errorf("Getting the return result timeout [%d]ms", timeoutMs)
}
type SyncExecuteDBResult struct {
sres chan *DBResultEx
err chan error
}
func (slf *DBModule) OnRun() bool {
if slf.db != nil {
slf.db.Ping()
func (slf *SyncExecuteDBResult) Get(timeoutMs int) (*DBResultEx, error) {
timerC := time.NewTicker(time.Millisecond * time.Duration(timeoutMs)).C
select {
case <-timerC:
break
case err := <-slf.err:
dataset := <-slf.sres
return dataset, err
}
time.Sleep(time.Second * 5)
return true
return nil, fmt.Errorf("Getting the return result timeout [%d]ms", timeoutMs)
}
func (slf *DBModule) Init(maxConn int, url string, userName string, password string, dbname string) error {
func (slf *DBModule) RunPing() {
for {
select {
case <-slf.pingCoroutine.pintExit:
log.Error("RunPing stopping %s...", fmt.Sprintf("%T", slf))
return
case <-slf.pingCoroutine.tickerPing.C:
if slf.db != nil {
slf.db.Ping()
}
}
}
}
func (slf *DBModule) Init(maxConn, executeNum int, url string, userName string, password string, dbname string) error {
slf.url = url
slf.maxconn = maxConn
slf.username = userName
slf.password = password
slf.dbname = dbname
slf.syncExecuteFun = make(chan SyncFun, MAX_EXECUTE_FUN)
slf.syncCoroutineNum = executeNum
if executeNum <= 0 {
return fmt.Errorf("executeNum mast more than zero:%d", executeNum)
}
slf.executeList = []DBExecute{}
for i := 0; i < executeNum; i++ {
itemInfo := DBExecute{syncExecuteFun:make(chan SyncFun, MAX_EXECUTE_FUN), syncExecuteExit:make(chan bool, 1)}
slf.executeList = append(slf.executeList, itemInfo)
}
slf.pingCoroutine = PingExecute{tickerPing : time.NewTicker(5*time.Second), pintExit : make(chan bool, 1)}
rand.Seed(time.Now().Unix())
return slf.Connect(slf.maxconn)
}
func (slf *DBModule) OnInit() error {
for i := 0; i < slf.syncCoroutineNum; i++ {
go slf.RunExecuteDBCoroutine()
go slf.RunExecuteDBCoroutine(i)
}
go slf.RunPing()
return nil
}
func (slf *DBModule) OnRelease() {
for i := 0; i < slf.syncCoroutineNum; i++ {
close(slf.executeList[i].syncExecuteExit)
}
}
//Close ...
func (slf *DBResult) Close() {
if slf.res != nil {
@@ -316,8 +387,6 @@ func (slf *DBModule) Connect(maxConn int) error {
db.SetMaxIdleConns(maxConn)
db.SetConnMaxLifetime(time.Second * 90)
slf.syncCoroutineNum = maxConn
return nil
}
@@ -335,19 +404,6 @@ func (slf *SyncDBResult) Get(timeoutMs int) DBResult {
}
}
func (slf *SyncQueryDBResultEx) Get(timeoutMs int) (*DataSetList, error) {
timerC := time.NewTicker(time.Millisecond * time.Duration(timeoutMs)).C
select {
case <-timerC:
break
case err := <-slf.err:
dataset := <-slf.sres
return dataset, err
}
return nil, fmt.Errorf("Getting the return result timeout [%d]ms", timeoutMs)
}
func (slf *DBModule) CheckArgs(args ...interface{}) error {
for _, val := range args {
if reflect.TypeOf(val).Kind() == reflect.String {
@@ -395,20 +451,20 @@ func (slf *DBModule) CheckArgs(args ...interface{}) error {
func (slf *DBModule) Query(query string, args ...interface{}) DBResult {
if slf.CheckArgs(args) != nil {
ret := DBResult{}
service.GetLogger().Printf(service.LEVER_ERROR, "CheckArgs is error :%s", query)
log.Error("CheckArgs is error :%s", query)
ret.Err = fmt.Errorf("CheckArgs is error!")
return ret
}
if slf.db == nil {
ret := DBResult{}
service.GetLogger().Printf(service.LEVER_ERROR, "cannot connect database:%s", query)
log.Error("cannot connect database:%s", query)
ret.Err = fmt.Errorf("cannot connect database!")
return ret
}
rows, err := slf.db.Query(query, args...)
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "Query:%s(%v)", query, err)
log.Error("Query:%s(%v)", query, err)
}
return DBResult{
@@ -425,12 +481,12 @@ func (slf *DBModule) QueryEx(query string, args ...interface{}) (*DataSetList, e
datasetList.blur = true
if slf.CheckArgs(args) != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "CheckArgs is error :%s", query)
log.Error("CheckArgs is error :%s", query)
return &datasetList, fmt.Errorf("CheckArgs is error!")
}
if slf.db == nil {
service.GetLogger().Printf(service.LEVER_ERROR, "cannot connect database:%s", query)
log.Error("cannot connect database:%s", query)
return &datasetList, fmt.Errorf("cannot connect database!")
}
@@ -439,10 +495,10 @@ func (slf *DBModule) QueryEx(query string, args ...interface{}) (*DataSetList, e
TimeFuncPass := time.Since(TimeFuncStart)
if slf.IsPrintTimeLog(TimeFuncPass) {
service.GetLogger().Printf(service.LEVER_INFO, "DBModule QueryEx Time %s , Query :%s , args :%+v", TimeFuncPass, query, args)
log.Error("DBModule QueryEx Time %s , Query :%s , args :%+v", TimeFuncPass, query, args)
}
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "Query:%s(%v)", query, err)
log.Error("Query:%s(%v)", query, err)
if rows != nil {
rows.Close()
}
@@ -483,7 +539,7 @@ func (slf *DBModule) QueryEx(query string, args ...interface{}) (*DataSetList, e
if hasRet == false {
if rows.Err() != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "Query:%s(%+v)", query, rows)
log.Error( "Query:%s(%+v)", query, rows)
}
break
}
@@ -493,13 +549,18 @@ func (slf *DBModule) QueryEx(query string, args ...interface{}) (*DataSetList, e
}
// SyncQuery ...
func (slf *DBModule) SyncQuery(query string, args ...interface{}) SyncQueryDBResultEx {
func (slf *DBModule) SyncQuery(queryHas int, query string, args ...interface{}) SyncQueryDBResultEx {
ret := SyncQueryDBResultEx{
sres: make(chan *DataSetList, 1),
err: make(chan error, 1),
}
if len(slf.syncExecuteFun) >= MAX_EXECUTE_FUN {
chanIndex := queryHas % len(slf.executeList)
if chanIndex < 0 {
chanIndex = rand.Intn(len(slf.executeList))
}
if len(slf.executeList[chanIndex].syncExecuteFun) >= MAX_EXECUTE_FUN {
dbret := DataSetList{}
ret.err <- fmt.Errorf("chan is full,sql:%s", query)
ret.sres <- &dbret
@@ -507,7 +568,7 @@ func (slf *DBModule) SyncQuery(query string, args ...interface{}) SyncQueryDBRes
return ret
}
slf.syncExecuteFun <- func() {
slf.executeList[chanIndex].syncExecuteFun <- func() {
rsp, err := slf.QueryEx(query, args...)
ret.err <- err
ret.sres <- rsp
@@ -516,16 +577,35 @@ func (slf *DBModule) SyncQuery(query string, args ...interface{}) SyncQueryDBRes
return ret
}
func (slf *DBModule) AsyncQuery(call SyncDBResultExCallBack, queryHas int, query string, args ...interface{}) error {
chanIndex := queryHas % len(slf.executeList)
if chanIndex < 0 {
chanIndex = rand.Intn(len(slf.executeList))
}
if len(slf.executeList[chanIndex].syncExecuteFun) >= MAX_EXECUTE_FUN {
return fmt.Errorf("chan is full,sql:%s", query)
}
slf.executeList[chanIndex].syncExecuteFun <- func() {
rsp, err := slf.QueryEx(query, args...)
call(rsp, err)
return
}
return nil
}
// Exec ...
func (slf *DBModule) Exec(query string, args ...interface{}) (*DBResultEx, error) {
ret := &DBResultEx{}
if slf.db == nil {
service.GetLogger().Printf(service.LEVER_ERROR, "cannot connect database:%s", query)
log.Error("cannot connect database:%s", query)
return ret, fmt.Errorf("cannot connect database!")
}
if slf.CheckArgs(args) != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "CheckArgs is error :%s", query)
log.Error("CheckArgs is error :%s", query)
//return ret, fmt.Errorf("cannot connect database!")
return ret, fmt.Errorf("CheckArgs is error!")
}
@@ -534,10 +614,10 @@ func (slf *DBModule) Exec(query string, args ...interface{}) (*DBResultEx, error
res, err := slf.db.Exec(query, args...)
TimeFuncPass := time.Since(TimeFuncStart)
if slf.IsPrintTimeLog(TimeFuncPass) {
service.GetLogger().Printf(service.LEVER_INFO, "DBModule QueryEx Time %s , Query :%s , args :%+v", TimeFuncPass, query, args)
log.Error("DBModule QueryEx Time %s , Query :%s , args :%+v", TimeFuncPass, query, args)
}
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "Exec:%s(%v)", query, err)
log.Error("Exec:%s(%v)", query, err)
return nil, err
}
@@ -548,18 +628,23 @@ func (slf *DBModule) Exec(query string, args ...interface{}) (*DBResultEx, error
}
// SyncExec ...
func (slf *DBModule) SyncExec(query string, args ...interface{}) *SyncExecuteDBResult {
func (slf *DBModule) SyncExec(queryHas int, query string, args ...interface{}) *SyncExecuteDBResult {
ret := &SyncExecuteDBResult{
sres: make(chan *DBResultEx, 1),
err: make(chan error, 1),
}
if len(slf.syncExecuteFun) >= MAX_EXECUTE_FUN {
chanIndex := queryHas % len(slf.executeList)
if chanIndex < 0 {
chanIndex = rand.Intn(len(slf.executeList))
}
if len(slf.executeList[chanIndex].syncExecuteFun) >= MAX_EXECUTE_FUN {
ret.err <- fmt.Errorf("chan is full,sql:%s", query)
return ret
}
slf.syncExecuteFun <- func() {
slf.executeList[chanIndex].syncExecuteFun <- func() {
rsp, err := slf.Exec(query, args...)
if err != nil {
ret.err <- err
@@ -573,19 +658,41 @@ func (slf *DBModule) SyncExec(query string, args ...interface{}) *SyncExecuteDBR
return ret
}
func (slf *DBModule) RunExecuteDBCoroutine() {
slf.WaitGroup.Add(1)
defer slf.WaitGroup.Done()
func (slf *DBModule) AsyncExec(call SyncDBResultExCallBack, queryHas int, query string, args ...interface{}) error {
chanIndex := queryHas % len(slf.executeList)
if chanIndex < 0 {
chanIndex = rand.Intn(len(slf.executeList))
}
if len(slf.executeList[chanIndex].syncExecuteFun) >= MAX_EXECUTE_FUN {
return fmt.Errorf("chan is full,sql:%s", query)
}
slf.executeList[chanIndex].syncExecuteFun <- func() {
rsp, err := slf.Exec(query, args...)
data := DataSetList{tag:"json", blur:true, dataSetList:[]DBResultEx{}}
data.dataSetList = append(data.dataSetList, *rsp)
call(&data, err)
return
}
return nil
}
func (slf *DBModule) RunExecuteDBCoroutine(has int) {
slf.waitGroup.Add(1)
defer slf.waitGroup.Done()
for {
select {
case <-slf.ExitChan:
service.GetLogger().Printf(LEVER_WARN, "stopping module %s...", fmt.Sprintf("%T", slf))
case <-slf.executeList[has].syncExecuteExit:
log.Error("stopping module %s...", fmt.Sprintf("%T", slf))
return
case fun := <-slf.syncExecuteFun:
case fun := <-slf.executeList[has].syncExecuteFun:
fun()
}
}
}
func (slf *DataSetList) UnMarshal(args ...interface{}) error {
@@ -740,7 +847,7 @@ func (slf *DBModule) Begin() (*Tx, error) {
var txDBMoudule Tx
txdb, err := slf.db.Begin()
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "Begin error:%s", err.Error())
log.Error("Begin error:%s", err.Error())
return &txDBMoudule, err
}
txDBMoudule.tx = txdb
@@ -805,21 +912,21 @@ func (slf *Tx) CheckArgs(args ...interface{}) error {
func (slf *Tx) Query(query string, args ...interface{}) DBResult {
if slf.CheckArgs(args) != nil {
ret := DBResult{}
service.GetLogger().Printf(service.LEVER_ERROR, "CheckArgs is error :%s", query)
log.Error("CheckArgs is error :%s", query)
ret.Err = fmt.Errorf("CheckArgs is error!")
return ret
}
if slf.tx == nil {
ret := DBResult{}
service.GetLogger().Printf(service.LEVER_ERROR, "cannot connect database:%s", query)
log.Error("cannot connect database:%s", query)
ret.Err = fmt.Errorf("cannot connect database!")
return ret
}
rows, err := slf.tx.Query(query, args...)
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "Tx Query:%s(%v)", query, err)
log.Error("Tx Query:%s(%v)", query, err)
}
return DBResult{
@@ -845,12 +952,12 @@ func (slf *Tx) QueryEx(query string, args ...interface{}) (*DataSetList, error)
datasetList.blur = true
if slf.CheckArgs(args) != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "CheckArgs is error :%s", query)
log.Error("CheckArgs is error :%s", query)
return &datasetList, fmt.Errorf("CheckArgs is error!")
}
if slf.tx == nil {
service.GetLogger().Printf(service.LEVER_ERROR, "cannot connect database:%s", query)
log.Error("cannot connect database:%s", query)
return &datasetList, fmt.Errorf("cannot connect database!")
}
@@ -859,10 +966,10 @@ func (slf *Tx) QueryEx(query string, args ...interface{}) (*DataSetList, error)
TimeFuncPass := time.Since(TimeFuncStart)
if slf.IsPrintTimeLog(TimeFuncPass) {
service.GetLogger().Printf(service.LEVER_INFO, "Tx QueryEx Time %s , Query :%s , args :%+v", TimeFuncPass, query, args)
log.Error("Tx QueryEx Time %s , Query :%s , args :%+v", TimeFuncPass, query, args)
}
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "Tx Query:%s(%v)", query, err)
log.Error("Tx Query:%s(%v)", query, err)
if rows != nil {
rows.Close()
}
@@ -903,7 +1010,7 @@ func (slf *Tx) QueryEx(query string, args ...interface{}) (*DataSetList, error)
if hasRet == false {
if rows.Err() != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "Tx Query:%s(%+v)", query, rows)
log.Error("Tx Query:%s(%+v)", query, rows)
}
break
}
@@ -916,12 +1023,12 @@ func (slf *Tx) QueryEx(query string, args ...interface{}) (*DataSetList, error)
func (slf *Tx) Exec(query string, args ...interface{}) (*DBResultEx, error) {
ret := &DBResultEx{}
if slf.tx == nil {
service.GetLogger().Printf(service.LEVER_ERROR, "cannot connect database:%s", query)
log.Error("cannot connect database:%s", query)
return ret, fmt.Errorf("cannot connect database!")
}
if slf.CheckArgs(args) != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "CheckArgs is error :%s", query)
log.Error("CheckArgs is error :%s", query)
//return ret, fmt.Errorf("cannot connect database!")
return ret, fmt.Errorf("CheckArgs is error!")
}
@@ -930,10 +1037,10 @@ func (slf *Tx) Exec(query string, args ...interface{}) (*DBResultEx, error) {
res, err := slf.tx.Exec(query, args...)
TimeFuncPass := time.Since(TimeFuncStart)
if slf.IsPrintTimeLog(TimeFuncPass) {
service.GetLogger().Printf(service.LEVER_INFO, "Tx QueryEx Time %s , Query :%s , args :%+v", TimeFuncPass, query, args)
log.Error("Tx QueryEx Time %s , Query :%s , args :%+v", TimeFuncPass, query, args)
}
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "Tx Exec:%s(%v)", query, err)
log.Error("Tx Exec:%s(%v)", query, err)
return nil, err
}

89
sysmodule/DBModule_test.go
View File

@@ -1,89 +0,0 @@
package sysmodule_test
import (
"fmt"
"sync"
"testing"
"github.com/duanhf2012/origin/sysmodule"
_ "github.com/go-sql-driver/mysql"
)
func TestDBModule(t *testing.T) {
db := sysmodule.DBModule{}
db.ExitChan = make(chan bool)
db.WaitGroup = new(sync.WaitGroup)
// db.Init(100, "192.168.0.5:3306", "root", "Root!!2018", "QuantFundsDB")
db.Init(100, "127.0.0.1:3306", "root", "zgh50221", "rebort_message")
db.OnInit()
tx, err := db.GetTx()
if err != nil {
fmt.Println("err 1", err)
return
}
res, err := tx.QueryEx("select id as Id, info_type as InfoType, info_type_Name as InfoTypeName from tbl_info_type where id >= 1")
if err != nil {
fmt.Println("err 2", err)
tx.Rollback()
return
}
out := []struct {
Id int64
InfoType string
InfoTypeName string
}{}
err = res.UnMarshal(&out)
if err != nil {
fmt.Println("err 3", err)
tx.Rollback()
return
}
fmt.Println(out)
_, err = tx.Exec("insert into tbl_info_type(info_type, info_type_name) VALUES (?, ?)", "4", "weibo")
if err != nil {
fmt.Println("err 4", err)
tx.Rollback()
return
}
_, err = tx.Exec("update tbl_info_type set info_types = ? Where id = ?", "5", 0)
if err != nil {
fmt.Println("err 4", err)
tx.Rollback()
return
}
tx.Commit()
// res, err := db.QueryEx("select * from tbl_fun_heelthrow where id >= 1")
// if err != nil {
// t.Error(err)
// }
// out := []struct {
// Addtime int64 `json:"addtime"`
// Tname string `json:"tname"`
// Uuid string `json:"uuid,omitempty"`
// AAAA string `json:"xxx"`
// }{}
// err = res.UnMarshal(&out)
// if err != nil {
// t.Error(err)
// }
// sres := db.SyncQuery("select * from tbl_fun_heelthrow where id >= 1")
// res, err = sres.Get(2000)
// if err != nil {
// t.Error(err)
// }
// out2 := []struct {
// Addtime int64 `json:"addtime"`
// Tname string `json:"tname"`
// Uuid string `json:"uuid,omitempty"`
// AAAA string `json:"xxx"`
// }{}
// err = res.UnMarshal(&out2)
// if err != nil {
// t.Error(err)
// }
}

4
sysmodule/HttpClientPoolModule.go
View File

@@ -10,11 +10,11 @@ import (
"net/url"
"time"
"github.com/duanhf2012/origin/service"
"github.com/duanhf2012/originnet/service"
)
type HttpClientPoolModule struct {
service.BaseModule
service.Module
client *http.Client
}

29
sysmodule/HttpClientPoolModule_test.go
View File

@@ -1,29 +0,0 @@
package sysmodule_test
import (
"fmt"
"net/http"
"testing"
"github.com/duanhf2012/origin/sysmodule"
)
func TestHttpClientPoolModule(t *testing.T) {
c := sysmodule.HttpClientPoolModule{}
c.Init(10, "")
rsp := c.Request(http.MethodGet, "https://www.baidu.com/", nil, nil)
fmt.Println(rsp.Err)
fmt.Println(rsp.Header)
fmt.Println(rsp.StatusCode)
fmt.Println(rsp.Status)
fmt.Println(string(rsp.Body))
srsp := c.SyncRequest(http.MethodGet, "https://www.baidu.com/", nil, nil)
rsp1 := srsp.Get(1)
fmt.Println(rsp1.Err)
fmt.Println(rsp1.Header)
fmt.Println(rsp1.StatusCode)
fmt.Println(rsp1.Status)
fmt.Println(string(rsp1.Body))
}

190
sysmodule/LogModule.go
View File

@@ -1,190 +0,0 @@
package sysmodule
import (
"fmt"
"log"
"os"
"path/filepath"
"runtime"
"strings"
"sync"
"time"
"github.com/duanhf2012/origin/service"
)
const (
maxLinesInLog = 100000 //一个日志文件最多只写这么多行
)
//等级从低到高
const (
LEVER_UNKNOW = 0
LEVER_DEBUG = 1
LEVER_INFO = 2
LEVER_WARN = 3
LEVER_ERROR = 4
LEVER_FATAL = 5
LEVEL_MAX = 6
)
var LogPrefix = [LEVEL_MAX]string{"[UNKNOW]", "[DEBUG]", "[INFO ]", "[WARN ]", "[ERROR]", "[FATAL]"}
type ILogger interface {
Printf(level uint, format string, v ...interface{})
Print(level uint, v ...interface{})
SetLogLevel(level uint)
}
type FunListenLog func(uint, string)
type LogModule struct {
service.BaseModule
currentDay int64
lines int64
logfilename string
logger [LEVEL_MAX]*log.Logger
logFile *os.File
openLevel uint
locker sync.Mutex
calldepth int
listenFun FunListenLog
}
func (slf *LogModule) GetCurrentFileName() string {
now := time.Now()
fpath := filepath.Join("logs")
os.MkdirAll(fpath, os.ModePerm)
y, m, d := now.Date()
h := now.Hour()
mm := now.Minute()
mm -= mm % 15 //15分钟内使用同一个日志文件
dt := y*10000 + int(m)*100 + d
tm := h*100 + mm
fname := fmt.Sprintf("%s-%d-%d.log", slf.logfilename, dt, tm)
ret := filepath.Join(fpath, fname)
return ret
}
//检查是否需要切换新的日志文件
func (slf *LogModule) CheckAndGenFile(fileline string) (newFile bool) {
now := time.Now()
nowDate := int64(now.Day())
slf.locker.Lock()
isNewDay := nowDate != slf.currentDay
slf.lines++
if isNewDay || slf.lines > maxLinesInLog {
// if time.Now().Day() == slf.currentDay {
// slf.locker.Unlock()
// return
// }
//fmt.Println("new log file", slf.currentDay, nowDate, isNewDay, slf.lines, maxLinesInLog)
slf.currentDay = nowDate
slf.lines = 1
newFile = true
if slf.logFile != nil {
slf.logFile.Close()
}
var err error
filename := slf.GetCurrentFileName()
slf.logFile, err = os.OpenFile(filename, os.O_RDWR|os.O_CREATE|os.O_APPEND, os.ModePerm)
if err != nil {
fmt.Printf("create log file %+v error!", filename)
slf.locker.Unlock()
return false
}
for level := 0; level < LEVEL_MAX; level++ {
slf.logger[level] = log.New(slf.logFile, LogPrefix[level], log.Lshortfile|log.LstdFlags)
}
}
slf.locker.Unlock()
return newFile
}
func (slf *LogModule) Init(logFilePrefixName string, openLevel uint) {
slf.currentDay = 0
slf.logfilename = logFilePrefixName
slf.openLevel = openLevel
slf.calldepth = 3
}
func (slf *LogModule) SetListenLogFunc(listenFun FunListenLog) {
slf.listenFun = listenFun
}
func (slf *LogModule) GetLoggerByLevel(level uint) *log.Logger {
if level >= LEVEL_MAX {
level = 0
}
return slf.logger[level]
}
func (slf *LogModule) Printf(level uint, format string, v ...interface{}) {
if level < slf.openLevel {
return
}
_, file, line, ok := runtime.Caller(slf.calldepth - 1)
if !ok {
file = "???"
line = 0
}
fileLine := fmt.Sprintf(" %s:%d: ", file, line)
slf.CheckAndGenFile(fileLine)
logContents := fmt.Sprintf(format, v...)
slf.doPutLog(level, fileLine, logContents)
}
func (slf *LogModule) Print(level uint, v ...interface{}) {
if level < slf.openLevel {
return
}
_, file, line, ok := runtime.Caller(slf.calldepth - 1)
if !ok {
file = "???"
line = 0
}
fileLine := fmt.Sprintf(" %s:%d: ", file, line)
slf.CheckAndGenFile(fileLine)
logContents := fmt.Sprint(v...)
slf.doPutLog(level, fileLine, logContents)
}
//最终写日志的接口
func (slf *LogModule) doPutLog(level uint, fileLine, logContents string) {
if slf.openLevel == LEVER_DEBUG || slf.listenFun != nil {
strlog := fmt.Sprintf("%s %s %s", LogPrefix[level], time.Now().Format("2006-01-02 15:04:05"), logContents)
if slf.openLevel == LEVER_DEBUG {
fmt.Println(strlog)
}
if slf.listenFun != nil {
fline := fileLine
if idx := strings.LastIndex(fileLine, "/"); idx >= 0 {
fline = fileLine[idx+1:]
}
ft := fline + " " + strlog
slf.listenFun(level, fmt.Sprintf(ft))
}
}
slf.GetLoggerByLevel(level).Output(slf.calldepth+1, logContents)
}
func (slf *LogModule) AppendCallDepth(calldepth int) {
slf.calldepth += calldepth
}
func (slf *LogModule) SetLogLevel(level uint) {
slf.openLevel = level
}

95
sysmodule/RedisModule.go
View File

@@ -5,9 +5,10 @@ import (
"encoding/json"
"errors"
"fmt"
"github.com/duanhf2012/originnet/log"
"time"
"github.com/duanhf2012/origin/service"
"github.com/duanhf2012/originnet/service"
"github.com/gomodule/redigo/redis"
)
@@ -39,7 +40,7 @@ func (slf *RetMapString) Get() (error, map[string]bool) {
type Func func()
type RedisModule struct {
service.BaseModule
service.Module
redispool *redis.Pool
redisTask chan Func
}
@@ -71,7 +72,7 @@ func (slf *RedisModule) Init(redisCfg *ConfigRedis) {
}
c, err := redis.Dial("tcp", redisServer, opt...)
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "Connect redis fail reason:%v", err)
log.Error("Connect redis fail reason:%v", err)
return nil, err
}
@@ -84,7 +85,7 @@ func (slf *RedisModule) Init(redisCfg *ConfigRedis) {
}
_, err := c.Do("PING")
if err != nil {
service.GetLogger().Printf(service.LEVER_WARN, "Do PING fail reason:%v", err)
log.Error("Do PING fail reason:%v", err)
return err
}
return err
@@ -107,7 +108,7 @@ func (slf *RedisModule) RunAnsyTask() {
func (slf *RedisModule) GoTask(fc Func) error {
if len(slf.redisTask) >= MAX_TASK_CHANNEL {
service.GetLogger().Printf(service.LEVER_ERROR, "Redis task channel recover max.")
log.Error("Redis task channel recover max.")
return fmt.Errorf("Redis task channel recover max.")
}
@@ -118,19 +119,19 @@ func (slf *RedisModule) GoTask(fc Func) error {
// GetConn ...
func (slf *RedisModule) getConn() (redis.Conn, error) {
if slf.redispool == nil {
service.GetLogger().Printf(service.LEVER_FATAL, "Not Init RedisModule")
log.Error("Not Init RedisModule")
return nil, fmt.Errorf("Not Init RedisModule")
}
conn := slf.redispool.Get()
if conn == nil {
service.GetLogger().Printf(service.LEVER_ERROR, "Cannot get connection")
log.Error("Cannot get connection")
return nil, fmt.Errorf("Cannot get connection")
}
if conn.Err() != nil {
err := conn.Err()
if err != nil {
service.GetLogger().Printf(service.LEVER_WARN, "Get Conn have error,reason:%v", err)
log.Error("Get Conn have error,reason:%v", err)
}
conn.Close()
return nil, err
@@ -148,7 +149,7 @@ func (slf *RedisModule) TestPingRedis() error {
err = slf.redispool.TestOnBorrow(conn, time.Now())
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "TestOnBorrow fail,reason:%v", err)
log.Error("TestOnBorrow fail,reason:%v", err)
return err
}
@@ -222,7 +223,7 @@ func (slf *RedisModule) GoSetStringJSONExpire(key string, val interface{}, expir
slf.GoSetStringExpire(key, string(temp), expire, retErr)
return nil
} else {
service.GetLogger().Printf(service.LEVER_ERROR, "GoSetStringJSONExpire fail,reason:%v", err)
log.Error("GoSetStringJSONExpire fail,reason:%v", err)
retErr.resultChan <- err
}
return err
@@ -248,14 +249,14 @@ func (slf *RedisModule) setStringByExpire(key, value, expire string) error {
}
if retErr != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "setStringByExpire fail,reason:%v", retErr)
log.Error("setStringByExpire fail,reason:%v", retErr)
return retErr
}
_, ok := ret.(string)
if !ok {
retErr = errors.New("setStringByExpire redis data is error")
service.GetLogger().Printf(service.LEVER_ERROR, "setStringByExpire redis data is error")
log.Error("setStringByExpire redis data is error")
return retErr
}
@@ -301,7 +302,7 @@ func (slf *RedisModule) GoSetMuchStringExpire(mapInfo map[string]string, expire
func (slf *RedisModule) setMuchStringByExpire(mapInfo map[string]string, expire string) error {
if len(mapInfo) <= 0 {
service.GetLogger().Printf(service.LEVER_ERROR, "setMuchStringByExpire Info Is Empty")
log.Error("setMuchStringByExpire Info Is Empty")
return errors.New("setMuchStringByExpire Info Is Empty")
}
@@ -324,7 +325,7 @@ func (slf *RedisModule) setMuchStringByExpire(mapInfo map[string]string, expire
_, err = conn.Do("EXEC")
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "setMuchStringByExpire fail,reason:%v", err)
log.Error("setMuchStringByExpire fail,reason:%v", err)
}
return err
@@ -341,7 +342,7 @@ func (slf *RedisModule) GetString(key string) (string, error) {
ret, err := conn.Do("GET", key)
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "GetString fail,reason:%v", err)
log.Error("GetString fail,reason:%v", err)
return "", err
}
@@ -353,7 +354,7 @@ func (slf *RedisModule) GetString(key string) (string, error) {
str, ok := ret.([]byte)
if !ok {
err = errors.New("GetString redis data is error")
service.GetLogger().Printf(service.LEVER_ERROR, "GetString redis data is error")
log.Error("GetString redis data is error")
return "", err
}
@@ -371,7 +372,7 @@ func (slf *RedisModule) GetStringJSON(key string, st interface{}) error {
ret, err := conn.Do("GET", key)
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "GetStringJSON fail,reason:%v", err)
log.Error("GetStringJSON fail,reason:%v", err)
return err
}
@@ -383,12 +384,12 @@ func (slf *RedisModule) GetStringJSON(key string, st interface{}) error {
str, ok := ret.([]byte)
if !ok {
err = errors.New("GetStringJSON redis data is error!")
service.GetLogger().Printf(service.LEVER_ERROR, "GetStringJSON redis data is error!")
log.Error("GetStringJSON redis data is error!")
return err
}
if err = json.Unmarshal(str, st); err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "GetStringJSON fail json.Unmarshal is error:%s,%s,reason:%v", key, string(str), err)
log.Error("GetStringJSON fail json.Unmarshal is error:%s,%s,reason:%v", key, string(str), err)
return err
}
@@ -413,7 +414,7 @@ func (slf *RedisModule) GetMuchString(keys []string) (retMap map[string]string,
// 开始Send数据
err = conn.Send("MULTI")
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "GetMuchString fail %v", err)
log.Error("GetMuchString fail %v", err)
return nil, err
}
for _, val := range keys {
@@ -423,7 +424,7 @@ func (slf *RedisModule) GetMuchString(keys []string) (retMap map[string]string,
ret, err := conn.Do("EXEC")
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "GetMuchString fail %v", err)
log.Error("GetMuchString fail %v", err)
return
}
@@ -455,7 +456,7 @@ func (slf *RedisModule) GetMuchString(keys []string) (retMap map[string]string,
func (slf *RedisModule) GetMuchStringJSON(keys map[string]interface{}) error {
if len(keys) <= 0 {
err := errors.New("GetMuchStringJSON fail key is empty")
service.GetLogger().Printf(service.LEVER_ERROR, "GetMuchStringJSON fail key is empty")
log.Error("GetMuchStringJSON fail key is empty")
return err
}
conn, err := slf.getConn()
@@ -476,7 +477,7 @@ func (slf *RedisModule) GetMuchStringJSON(keys map[string]interface{}) error {
ret, err := conn.Do("EXEC")
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "GetMuchStringJSON fail, reason:%v", err)
log.Error("GetMuchStringJSON fail, reason:%v", err)
return err
}
@@ -496,7 +497,7 @@ func (slf *RedisModule) GetMuchStringJSON(keys map[string]interface{}) error {
err = json.Unmarshal(strVal, keys[tempKeys[index]])
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "GetMuchStringJSON Unmarshal fail, reason:%v", err)
log.Error("GetMuchStringJSON Unmarshal fail, reason:%v", err)
return err
}
}
@@ -514,7 +515,7 @@ func (slf *RedisModule) ExistsKey(key string) (bool, error) {
ret, err := conn.Do("EXISTS", key)
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "ExistsKey fail, reason:%v", err)
log.Error("ExistsKey fail, reason:%v", err)
return false, err
}
retValue, ok := ret.(int64)
@@ -537,7 +538,7 @@ func (slf *RedisModule) DelString(key string) error {
ret, err := conn.Do("DEL", key)
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "DelString fail, reason:%v", err)
log.Error("DelString fail, reason:%v", err)
return err
}
@@ -613,7 +614,7 @@ func (slf *RedisModule) DelMuchString(keys []string) (map[string]bool, error) {
ret, err := conn.Do("EXEC")
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "DelMuchString fail,reason:%v", err)
log.Error("DelMuchString fail,reason:%v", err)
return nil, err
}
@@ -652,7 +653,7 @@ func (slf *RedisModule) SetHash(redisKey, hashKey, value string) error {
_, retErr := conn.Do("HSET", redisKey, hashKey, value)
if retErr != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "SetHash fail,reason:%v", retErr)
log.Error("SetHash fail,reason:%v", retErr)
}
return retErr
@@ -686,7 +687,7 @@ func (slf *RedisModule) GetAllHashJSON(redisKey string) (map[string]string, erro
value, err := conn.Do("HGETALL", redisKey)
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "GetAllHashJSON fail,reason:%v", err)
log.Error("GetAllHashJSON fail,reason:%v", err)
return nil, err
}
@@ -697,7 +698,7 @@ func (slf *RedisModule) GetAllHashJSON(redisKey string) (map[string]string, erro
func (slf *RedisModule) GetHashValueByKey(redisKey string, fieldKey string) (string, error) {
if redisKey == "" || fieldKey == "" {
service.GetLogger().Printf(service.LEVER_ERROR, "GetHashValueByKey key is empty!")
log.Error("GetHashValueByKey key is empty!")
return "", errors.New("Key Is Empty")
}
conn, err := slf.getConn()
@@ -708,7 +709,7 @@ func (slf *RedisModule) GetHashValueByKey(redisKey string, fieldKey string) (str
value, err := conn.Do("HGET", redisKey, fieldKey)
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "GetHashValueByKey fail,reason:%v", err)
log.Error("GetHashValueByKey fail,reason:%v", err)
return "", err
}
if value == nil {
@@ -770,7 +771,7 @@ func (slf *RedisModule) SetMuchHashJSON(redisKey string, value map[string][]inte
// 执行命令
_, err = conn.Do("EXEC")
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "SetMuchHashJSON fail,reason:%v", err)
log.Error("SetMuchHashJSON fail,reason:%v", err)
}
return err
}
@@ -841,7 +842,7 @@ func (slf *RedisModule) DelMuchHash(redisKey string, hsahKey []string) error {
_, retErr := conn.Do("HDEL", arg...)
if retErr != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "DelMuchHash fail,reason:%v", retErr)
log.Error("DelMuchHash fail,reason:%v", retErr)
}
return retErr
}
@@ -883,7 +884,7 @@ func (slf *RedisModule) setList(key string, value []string, setType string) erro
}
_, retErr := conn.Do(setType, arg...)
if retErr != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "setList fail,reason:%v", retErr)
log.Error("setList fail,reason:%v", retErr)
}
return retErr
}
@@ -918,7 +919,7 @@ func (slf *RedisModule) SetListJSONLpush(key string, value interface{}) error {
tempVal := []string{string(temp)}
err = slf.setList(key, tempVal, "LPUSH")
} else {
service.GetLogger().Printf(service.LEVER_ERROR, "SetListJSONLpush fail,reason:%v", err)
log.Error("SetListJSONLpush fail,reason:%v", err)
}
return err
}
@@ -955,7 +956,7 @@ func (slf *RedisModule) SetMuchListJSONLpush(key string, value []interface{}) er
for _, val := range value {
temp, err := json.Marshal(val)
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "SetMuchListJSONLpush fail,reason:%v", err)
log.Error("SetMuchListJSONLpush fail,reason:%v", err)
return err
}
@@ -998,7 +999,7 @@ func (slf *RedisModule) SetListJSONRpush(key string, value interface{}) error {
tempVal := []string{string(temp)}
err = slf.setList(key, tempVal, "RPUSH")
} else {
service.GetLogger().Printf(service.LEVER_ERROR, "SetListJSONRpush fail,reason:%v", err)
log.Error("SetListJSONRpush fail,reason:%v", err)
}
return err
@@ -1052,7 +1053,7 @@ func (slf *RedisModule) Lrange(key string, start, end int) ([]string, error) {
reply, err := conn.Do("lrange", key, start, end)
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "SetListJSONRpush fail,reason:%v", err)
log.Error("SetListJSONRpush fail,reason:%v", err)
return nil, err
}
@@ -1069,7 +1070,7 @@ func (slf *RedisModule) GetListLen(key string) (int, error) {
reply, err := conn.Do("LLEN", key)
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "GetListLen fail,reason:%v", err)
log.Error("GetListLen fail,reason:%v", err)
return -1, err
}
return redis.Int(reply, err)
@@ -1086,7 +1087,7 @@ func (slf *RedisModule) RPOPListValue(key string) error {
_, err = conn.Do("RPOP", key, 100)
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "RPOPListValue fail,reason:%v", err)
log.Error("RPOPListValue fail,reason:%v", err)
return err
}
return nil
@@ -1103,7 +1104,7 @@ func (slf *RedisModule) LtrimList(key string, start, end int) error {
_, err = conn.Do("LTRIM", key, start, end)
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "LtrimListValue fail,reason:%v", err)
log.Error("LtrimListValue fail,reason:%v", err)
return err
}
return nil
@@ -1124,7 +1125,7 @@ func (slf *RedisModule) ZADDInsertJson(key string, score float64, value interfac
}
_, err = conn.Do("ZADD", key, score, JsonValue)
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "ZADDInsertJson fail,reason:%v", err)
log.Error("ZADDInsertJson fail,reason:%v", err)
return err
}
return nil
@@ -1140,7 +1141,7 @@ func (slf *RedisModule) ZADDInsert(key string, score float64, Data interface{})
_, err = conn.Do("ZADD", key, score, Data)
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "ZADDInsert fail,reason:%v", err)
log.Error("ZADDInsert fail,reason:%v", err)
return err
}
return nil
@@ -1444,13 +1445,13 @@ func (slf *RedisModule) HincrbyHashInt(redisKey, hashKey string, value int) erro
_, retErr := conn.Do("HINCRBY", redisKey, hashKey, value)
if retErr != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "HincrbyHashInt fail,reason:%v", retErr)
log.Error("HincrbyHashInt fail,reason:%v", retErr)
}
return retErr
}
func (slf *RedisModule) EXPlREInsert(key string, TTl int) error {
func (slf *RedisModule) EXPlREInsert(key string, TTl int) error {
conn, err := slf.getConn()
if err != nil {
return err
@@ -1459,7 +1460,7 @@ func (slf *RedisModule) HincrbyHashInt(redisKey, hashKey string, value int) erro
_, err = conn.Do("expire", key, TTl)
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "expire fail,reason:%v", err)
log.Error("expire fail,reason:%v", err)
return err
}
return nil
@@ -1486,7 +1487,7 @@ func (slf *RedisModule) Keys(key string) ([]string, error) {
ret, err := conn.Do("KEYS", key)
if err != nil {
service.GetLogger().Printf(service.LEVER_ERROR, "KEYS fail, reason:%v", err)
log.Error("KEYS fail, reason:%v", err)
return nil, err
}
retList, ok := ret.([]interface{})

116
sysmodule/RedisModule_test.go
View File

@@ -1,116 +0,0 @@
package sysmodule
import (
"fmt"
"testing"
"time"
)
type CTestJson struct {
A string
B string
}
func TestRedisModule(t *testing.T) {
var cfg ConfigRedis
var redsmodule RedisModule
cfg.IP = "127.0.0.1"
cfg.Password = ""
cfg.Port = "6379"
cfg.IdleTimeout = 2
cfg.MaxActive = 3
cfg.MaxIdle = 3
cfg.DbIndex = 15
redsmodule.Init(&cfg)
//GoSetString
var retErr RetError
redsmodule.GoSetString("testkey", "testvalue", &retErr)
ret1, err1 := redsmodule.GetString("testkey")
//GoSetStringExpire
redsmodule.GoSetStringExpire("setstring_key", "value11", "1", &retErr)
time.Sleep(time.Second * 2)
ret2, err2 := redsmodule.GetString("setstring_key")
fmt.Print(ret1, err1, retErr.Get(), "\n")
fmt.Print(ret2, err2, "\n")
//GoSetStringJSON
var testjson CTestJson
testjson.A = "A value"
testjson.B = "B value"
var retErr2 RetError
redsmodule.GoSetStringJSON("setstring_key_json", &testjson, &retErr)
//GoSetStringJSONExpire
redsmodule.GoSetStringJSONExpire("setstring_key_json_expire", &testjson, "10", &retErr2)
fmt.Print(retErr.Get(), "\n")
fmt.Print(retErr2.Get(), "\n")
var testjson2 CTestJson
err := redsmodule.GetStringJSON("setstring_key_json", &testjson2)
fmt.Print(err, "\n", testjson2, "\n")
//GoSetMuchString/GoSetMuchStringExpire
mapInfo := make(map[string]string)
mapInfo["A"] = "A_value"
mapInfo["B"] = "B_value"
mapInfo["C"] = "C_value"
redsmodule.GoSetMuchString(mapInfo, &retErr)
redsmodule.GoSetMuchStringExpire(mapInfo, "100", &retErr)
var keylist []string
keylist = append(keylist, "A", "C")
mapInfo, _ = redsmodule.GetMuchString(keylist)
fmt.Print(retErr.Get(), "\n", mapInfo, "\n")
//GoDelString
redsmodule.GoDelString("setstring_key_json", &retErr)
fmt.Print(retErr.Get(), "\n")
var retMapStr RetMapString
redsmodule.GoDelMuchString(keylist, &retMapStr)
err2, mapstr := retMapStr.Get()
fmt.Print(err2, "\n", mapstr, "\n")
//GoSetHash
redsmodule.GoSetHash("rediskey", "hashkey", "1111", &retErr)
ret, err := redsmodule.GetHashValueByKey("rediskey", "hashkey")
fmt.Print(retErr.Get(), ret, err)
//GoSetHashJSON
redsmodule.GoSetHashJSON("rediskey", "hashkey2", &testjson, &retErr)
fmt.Print(retErr.Get(), "\n")
//SetMuchHashJSON
var mapHashJson map[string][]interface{}
var listInterface []interface{}
mapHashJson = make(map[string][]interface{})
listInterface = append(listInterface, testjson, testjson)
mapHashJson["key3"] = listInterface
mapHashJson["key4"] = listInterface
redsmodule.GoSetMuchHashJSON("rediskey", mapHashJson, &retErr)
fmt.Print(retErr.Get(), "\n")
//GoDelHash
redsmodule.GoDelHash("rediskey", "hashkey", &retErr)
fmt.Print(retErr.Get(), "\n")
//GoDelMuchHash
var keys []string
keys = append(keys, "hashkey", "hashkey2")
redsmodule.GoDelMuchHash("rediskey", keys, &retErr)
fmt.Print(retErr.Get(), "\n")
//GoSetListLpush
redsmodule.GoSetListLpush("setlistKey", "list1", &retErr)
fmt.Print(retErr.Get(), "\n")
redsmodule.GoSetListLpush("setlistKey", "list2", &retErr)
fmt.Print(retErr.Get(), "\n")
redsmodule.GoSetListLpush("setlistKey", "list3", &retErr)
fmt.Print(retErr.Get(), "\n")
}

7
sysmodule/SystemModuleDef.go
View File

@@ -1,7 +0,0 @@
package sysmodule
//等级从低到高
const (
SYS_LOG = 1
SYS_TEST = 2
)

37
sysservice/logservice.go
View File

@@ -1,37 +0,0 @@
package sysservice
import (
"github.com/duanhf2012/origin/service"
"github.com/duanhf2012/origin/sysmodule"
)
type LogService struct {
service.BaseService
logmodule *sysmodule.LogModule
}
func (slf *LogService) InitLog(logservicename string, logFilePrefix string, openLevel uint) {
slf.SetServiceName(logservicename)
slf.logmodule = &sysmodule.LogModule{}
slf.logmodule.Init(logFilePrefix, openLevel)
}
func (slf *LogService) Printf(level uint, format string, v ...interface{}) {
slf.logmodule.Printf(level, format, v...)
}
func (slf *LogService) Print(level uint, v ...interface{}) {
slf.logmodule.Print(level, v...)
}
func (slf *LogService) AppendCallDepth(calldepth int) {
slf.logmodule.AppendCallDepth(calldepth)
}
func (slf *LogService) SetLogLevel(level uint) {
slf.logmodule.SetLogLevel(level)
}
func (slf *LogService) SetListenLogFunc(listenFun sysmodule.FunListenLog) {
slf.logmodule.SetListenLogFunc(listenFun)
}

511
sysservice/originhttp/httpserverervice.go
View File

@@ -1,511 +0,0 @@
package originhttp
import (
"encoding/json"
"errors"
"fmt"
"io"
"io/ioutil"
"net/http"
"os"
"reflect"
"runtime"
"strings"
"time"
"github.com/duanhf2012/origin/sysmodule"
"github.com/duanhf2012/origin/rpc"
"github.com/duanhf2012/origin/cluster"
"github.com/duanhf2012/origin/network"
"github.com/duanhf2012/origin/service"
"github.com/duanhf2012/origin/util/uuid"
)
type HttpRedirectData struct {
Url string
//Cookies map[string]string
CookieList []*http.Cookie
}
type HttpRequest struct {
Header http.Header
Body string
ParamStr string //http://127.0.0.1:7001/aaa/bbb?aa=1中的aa=1部分
mapParam map[string]string
URL string
//Req http.Request
}
type HttpRespone struct {
Respone []byte
RedirectData HttpRedirectData
//Resp http.ResponseWriter
}
type ServeHTTPRouterMux struct {
httpfiltrateList []HttpFiltrate
allowOrigin bool
}
type ControllerMapsType map[string]reflect.Value
type HttpServerService struct {
service.BaseService
httpserver network.HttpServer
port uint16
controllerMaps ControllerMapsType
certfile string
keyfile string
ishttps bool
serverHTTPMux ServeHTTPRouterMux
}
type RouterMatchData struct {
callpath string
matchURL string
routerType int8 //0表示函数调用 1表示静态资源
}
type RouterStaticResoutceData struct {
localpath string
method string
}
type HTTP_METHOD int
const (
METHOD_GET HTTP_METHOD = iota
METHOD_POST HTTP_METHOD = 1
//METHOD_PUT HTTP_METHOD = 2
)
var bPrintRequestTime bool
var postAliasUrl map[string]map[string]RouterMatchData //url地址，对应本service地址
var staticRouterResource map[string]RouterStaticResoutceData
func init() {
postAliasUrl = make(map[string]map[string]RouterMatchData)
postAliasUrl["GET"] = make(map[string]RouterMatchData)
postAliasUrl["POST"] = make(map[string]RouterMatchData)
staticRouterResource = make(map[string]RouterStaticResoutceData)
}
type HttpHandle func(request *HttpRequest, resp *HttpRespone) error
func AnalysisRouterUrl(url string) (string, error) {
//替换所有空格
url = strings.ReplaceAll(url, " ", "")
if len(url) <= 1 || url[0] != '/' {
return "", fmt.Errorf("url %s format is error!", url)
}
//去掉尾部的/
return strings.Trim(url, "/"), nil
}
func (slf *HttpRequest) Query(key string) (string, bool) {
if slf.mapParam == nil {
slf.mapParam = make(map[string]string)
//分析字符串
slf.ParamStr = strings.Trim(slf.ParamStr, "/")
paramStrList := strings.Split(slf.ParamStr, "&")
for _, val := range paramStrList {
Index := strings.Index(val, "=")
if Index >= 0 {
slf.mapParam[val[0:Index]] = val[Index+1:]
}
}
}
ret, ok := slf.mapParam[key]
return ret, ok
}
func Request(method HTTP_METHOD, url string, handle HttpHandle) error {
fnpath := runtime.FuncForPC(reflect.ValueOf(handle).Pointer()).Name()
sidx := strings.LastIndex(fnpath, "*")
if sidx == -1 {
return errors.New(fmt.Sprintf("http post func path is error, %s\n", fnpath))
}
eidx := strings.LastIndex(fnpath, "-fm")
if sidx == -1 {
return errors.New(fmt.Sprintf("http post func path is error, %s\n", fnpath))
}
callpath := fnpath[sidx+1 : eidx]
ridx := strings.LastIndex(callpath, ")")
if ridx == -1 {
return errors.New(fmt.Sprintf("http post func path is error, %s\n", fnpath))
}
hidx := strings.LastIndex(callpath, "HTTP_")
if hidx == -1 {
return errors.New(fmt.Sprintf("http post func not contain HTTP_, %s\n", fnpath))
}
callpath = strings.ReplaceAll(callpath, ")", "")
var r RouterMatchData
var matchURL string
var err error
r.routerType = 0
r.callpath = "_" + callpath
matchURL, err = AnalysisRouterUrl(url)
if err != nil {
return err
}
var strMethod string
if method == METHOD_GET {
strMethod = "GET"
} else if method == METHOD_POST {
strMethod = "POST"
} else {
return nil
}
postAliasUrl[strMethod][matchURL] = r
return nil
}
func Post(url string, handle HttpHandle) error {
return Request(METHOD_POST, url, handle)
}
func Get(url string, handle HttpHandle) error {
return Request(METHOD_GET, url, handle)
}
func (slf *HttpServerService) OnInit() error {
slf.serverHTTPMux = ServeHTTPRouterMux{}
slf.httpserver.Init(slf.port, &slf.serverHTTPMux, 10*time.Second, 10*time.Second)
if slf.ishttps == true {
slf.httpserver.SetHttps(slf.certfile, slf.keyfile)
}
return nil
}
func (slf *ServeHTTPRouterMux) SetAlowOrigin(allowOrigin bool) {
slf.allowOrigin = allowOrigin
}
func (slf *ServeHTTPRouterMux) ServeHTTP(w http.ResponseWriter, r *http.Request) {
if slf.allowOrigin == true {
if origin := r.Header.Get("Origin"); origin != "" {
w.Header().Set("Access-Control-Allow-Origin", "*")
w.Header().Set("Access-Control-Allow-Methods", "POST, GET, OPTIONS, PUT, DELETE")
w.Header().Set("Access-Control-Allow-Headers",
"Action, Module") //有使用自定义头 需要这个,Action, Module是例子
}
}
if r.Method == "OPTIONS" {
return
}
methodRouter, bok := postAliasUrl[r.Method]
if bok == false {
writeRespone(w, http.StatusNotFound, fmt.Sprint("Can not support method."))
return
}
//权限验证
var errRet error
for _, filter := range slf.httpfiltrateList {
ret := filter(r.URL.Path, w, r)
if ret == nil {
errRet = nil
break
} else {
errRet = ret
}
}
if errRet != nil {
writeRespone(w, http.StatusOK, errRet.Error())
return
}
url := strings.Trim(r.URL.Path, "/")
var strCallPath string
matchData, ok := methodRouter[url]
if ok == true {
strCallPath = matchData.callpath
} else {
//如果是资源处理
for k, v := range staticRouterResource {
idx := strings.Index(url, k)
if idx != -1 {
staticServer(k, v, w, r)
return
}
}
// 拼接得到rpc服务的名称
vstr := strings.Split(url, "/")
if len(vstr) < 2 {
writeRespone(w, http.StatusNotFound, "Cannot find path.")
return
}
strCallPath = "_" + vstr[0] + ".HTTP_" + vstr[1]
}
defer r.Body.Close()
msg, err := ioutil.ReadAll(r.Body)
if err != nil {
writeRespone(w, http.StatusBadRequest, "")
return
}
request := HttpRequest{r.Header, string(msg), r.URL.RawQuery, nil, r.URL.Path}
var resp HttpRespone
//resp.Resp = w
timeFuncStart := time.Now()
err = cluster.InstanceClusterMgr().Call(strCallPath, &request, &resp)
timeFuncPass := time.Since(timeFuncStart)
if bPrintRequestTime {
service.GetLogger().Printf(service.LEVER_INFO, "HttpServer Time : %s url : %s\n", timeFuncPass, strCallPath)
}
if err != nil {
writeRespone(w, http.StatusBadRequest, fmt.Sprint(err))
} else {
if resp.RedirectData.Url != "" {
resp.redirects(&w, r)
} else {
writeRespone(w, http.StatusOK, string(resp.Respone))
}
}
}
// CkResourceDir 检查静态资源文件夹路径
func SetStaticResource(method HTTP_METHOD, urlpath string, dirname string) error {
_, err := os.Stat(dirname)
if err != nil {
return err
}
matchURL, berr := AnalysisRouterUrl(urlpath)
if berr != nil {
return berr
}
var routerData RouterStaticResoutceData
if method == METHOD_GET {
routerData.method = "GET"
} else if method == METHOD_POST {
routerData.method = "POST"
} else {
return nil
}
routerData.localpath = dirname
staticRouterResource[matchURL] = routerData
return nil
}
func writeRespone(w http.ResponseWriter, status int, msg string) {
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
w.WriteHeader(status)
w.Write([]byte(msg))
}
type HttpFiltrate func(path string, w http.ResponseWriter, r *http.Request) error
func (slf *HttpServerService) AppendHttpFiltrate(fun HttpFiltrate) bool {
slf.serverHTTPMux.httpfiltrateList = append(slf.serverHTTPMux.httpfiltrateList, fun)
return false
}
func (slf *HttpServerService) OnRun() bool {
slf.httpserver.Start()
return false
}
func NewHttpServerService(port uint16) *HttpServerService {
http := new(HttpServerService)
http.port = port
return http
}
func (slf *HttpServerService) OnDestory() error {
return nil
}
func (slf *HttpServerService) OnSetupService(iservice service.IService) {
rpc.RegisterName(iservice.GetServiceName(), "HTTP_", iservice)
}
func (slf *HttpServerService) OnRemoveService(iservice service.IService) {
return
}
func (slf *HttpServerService) SetPrintRequestTime(isPrint bool) {
bPrintRequestTime = isPrint
}
func staticServer(routerUrl string, routerData RouterStaticResoutceData, w http.ResponseWriter, r *http.Request) {
upath := r.URL.Path
idx := strings.Index(upath, routerUrl)
subPath := strings.Trim(upath[idx+len(routerUrl):], "/")
destLocalPath := routerData.localpath + subPath
writeResp := func(status int, msg string) {
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
w.WriteHeader(status)
w.Write([]byte(msg))
}
switch r.Method {
//获取资源
case "GET":
//判断文件夹是否存在
_, err := os.Stat(destLocalPath)
if err == nil {
http.ServeFile(w, r, destLocalPath)
} else {
writeResp(http.StatusNotFound, "")
return
}
//上传资源
case "POST":
// 在这儿处理例外路由接口
/*
var errRet error
for _, filter := range slf.httpfiltrateList {
ret := filter(r.URL.Path, w, r)
if ret == nil {
errRet = nil
break
} else {
errRet = ret
}
}
if errRet != nil {
w.Write([]byte(errRet.Error()))
return
}*/
r.ParseMultipartForm(32 << 20) // max memory is set to 32MB
resourceFile, resourceFileHeader, err := r.FormFile("file")
if err != nil {
fmt.Println(err)
writeResp(http.StatusNotFound, err.Error())
return
}
defer resourceFile.Close()
//重新拼接文件名
imgFormat := strings.Split(resourceFileHeader.Filename, ".")
if len(imgFormat) < 2 {
writeResp(http.StatusNotFound, "not a file")
return
}
filePrefixName := uuid.Rand().HexEx()
fileName := filePrefixName + "." + imgFormat[len(imgFormat)-1]
//创建文件
localpath := fmt.Sprintf("%s%s", destLocalPath, fileName)
localfd, err := os.OpenFile(localpath, os.O_WRONLY|os.O_CREATE, 0666)
if err != nil {
fmt.Println(err)
writeResp(http.StatusNotFound, "upload fail")
return
}
defer localfd.Close()
io.Copy(localfd, resourceFile)
writeResp(http.StatusOK, upath+"/"+fileName)
}
}
func (slf *HttpServerService) GetMethod(strCallPath string) (*reflect.Value, error) {
value, ok := slf.controllerMaps[strCallPath]
if ok == false {
err := fmt.Errorf("not find api")
return nil, err
}
return &value, nil
}
func (slf *HttpServerService) SetHttps(certfile string, keyfile string) bool {
if certfile == "" || keyfile == "" {
return false
}
slf.ishttps = true
slf.certfile = certfile
slf.keyfile = keyfile
return true
}
//序列化后写入Respone
func (slf *HttpRespone) WriteRespne(v interface{}) error {
StrRet, retErr := json.Marshal(v)
if retErr != nil {
slf.Respone = []byte(`{"Code": 2,"Message":"service error"}`)
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "Json Marshal Error:%v\n", retErr)
} else {
slf.Respone = StrRet
}
return retErr
}
func (slf *HttpRespone) WriteRespones(Code int32, Msg string, Data interface{}) {
var StrRet string
//判断是否有错误码
if Code > 0 {
StrRet = fmt.Sprintf(`{"RCode": %d,"RMsg":"%s"}`, Code, Msg)
} else {
if Data == nil {
if Msg != "" {
StrRet = fmt.Sprintf(`{"RCode": 0,"RMsg":"%s"}`, Msg)
} else {
StrRet = `{"RCode": 0}`
}
} else {
if reflect.TypeOf(Data).Kind() == reflect.String {
StrRet = fmt.Sprintf(`{"RCode": %d , "Data": "%s"}`, Code, Data)
} else {
JsonRet, Err := json.Marshal(Data)
if Err != nil {
service.GetLogger().Printf(sysmodule.LEVER_ERROR, "common WriteRespone Json Marshal Err %+v", Data)
} else {
StrRet = fmt.Sprintf(`{"RCode": %d , "Data": %s}`, Code, JsonRet)
}
}
}
}
slf.Respone = []byte(StrRet)
}
func (slf *HttpRespone) Redirect(url string, cookieList []*http.Cookie) {
slf.RedirectData.Url = url
slf.RedirectData.CookieList = cookieList
}
func (slf *HttpRespone) redirects(w *http.ResponseWriter, req *http.Request) {
if slf.RedirectData.CookieList != nil {
for _, v := range slf.RedirectData.CookieList {
http.SetCookie(*w, v)
}
}
http.Redirect(*w, req, slf.RedirectData.Url,
// see @andreiavrammsd comment: often 307 > 301
http.StatusTemporaryRedirect)
}

73
sysservice/pprofservice.go
View File

@@ -1,73 +0,0 @@
package sysservice
import (
"encoding/json"
"fmt"
"runtime/pprof"
"time"
"github.com/duanhf2012/origin/service"
"github.com/duanhf2012/origin/sysservice/originhttp"
)
type PProfService struct {
service.BaseService
fisttime int
}
type ProfileData struct {
Name string
Count int
}
type Profilestruct struct {
Fisttime int
ProfileList []ProfileData
}
func (slf *PProfService) OnInit() error {
slf.fisttime = (int)(time.Now().UnixNano())
return nil
}
func (slf *PProfService) GetPprof() ([]byte, error) {
var pfiles Profilestruct
pfiles.Fisttime = slf.fisttime
for _, p := range pprof.Profiles() {
pfiles.ProfileList = append(pfiles.ProfileList, ProfileData{
Name: p.Name(),
Count: p.Count(),
})
}
return json.Marshal(pfiles)
}
func (slf *PProfService) HTTP_DebugPProf(request *originhttp.HttpRequest, resp *originhttp.HttpRespone) error {
var err error
resp.Respone, err = slf.GetPprof()
if err != nil {
resp.Respone = []byte(fmt.Sprint(err))
}
return nil
}
func (slf *PProfService) RPC_DebugPProf(arg *string, ret *Profilestruct) error {
ret.Fisttime = slf.fisttime
for _, p := range pprof.Profiles() {
ret.ProfileList = append(ret.ProfileList, ProfileData{
Name: p.Name(),
Count: p.Count(),
})
}
return nil
}
func (slf *PProfService) HTTP_Test(request *originhttp.HttpRequest, resp *originhttp.HttpRespone) error {
resp.Respone = []byte(request.Body)
return nil
}

167
sysservice/synccacheservice/synccacheservice.go
View File

@@ -1,167 +0,0 @@
package synccacheservice
import (
"encoding/json"
"errors"
"fmt"
"time"
"github.com/duanhf2012/origin/cluster"
"github.com/duanhf2012/origin/service"
"github.com/duanhf2012/origin/util"
)
const (
MAX_SYNC_DATA_CHAN_NUM = 10000
)
//CReportService ...
type CSyncCacheService struct {
service.BaseService
mapCache util.Map
syncQueue *util.SyncQueue
nodeIdList []int
syncDataChanList []chan *SyncCacheData
}
type SyncCacheData struct {
OperType int8 //0 表示添加或者更新 1表示删除
Key string
Val string
Wxpire int32 //ms
NodeIdList []int
reTryCount uint32
reTryTime int64
}
//OnInit ...
func (slf *CSyncCacheService) OnInit() error {
slf.syncQueue = util.NewSyncQueue()
var callServiceName string
slf.nodeIdList = cluster.InstanceClusterMgr().GetNodeList("CSyncCacheService.RPC_SyncString", &callServiceName, nil)
for _, nodeId := range slf.nodeIdList {
syncCacheData := make(chan *SyncCacheData, MAX_SYNC_DATA_CHAN_NUM)
slf.syncDataChanList = append(slf.syncDataChanList, syncCacheData)
go slf.syncRouter(nodeId, syncCacheData)
}
return nil
}
func (slf *CSyncCacheService) syncRouter(nodeId int, syncDataChan chan *SyncCacheData) error {
tryCount := 0
for {
select {
case <-slf.ExitChan:
break
case data := <-syncDataChan:
var ret int
cluster.CallNode(nodeId, "CSyncCacheService.RPC_SyncString", data, &ret)
if ret == 0 {
if tryCount < 3 {
time.Sleep(800 * time.Millisecond)
} else {
time.Sleep(1500 * time.Millisecond)
}
slf.tryPushSyncData(syncDataChan, data)
tryCount++
} else {
tryCount = 0
}
}
}
return nil
}
func (slf *CSyncCacheService) tryPushSyncData(syncDataChan chan *SyncCacheData, syncData *SyncCacheData) bool {
if len(syncDataChan) >= MAX_SYNC_DATA_CHAN_NUM {
return false
}
syncDataChan <- syncData
return true
}
func (slf *CSyncCacheService) RPC_SyncString(request *SyncCacheData, ret *int) error {
if request.OperType == 0 {
slf.mapCache.Set(request.Key, request.Val)
} else {
slf.mapCache.Del(request.Key)
}
*ret = 1
return nil
}
func SetStringJson(key string, val interface{}) error {
byteBuf, err := json.Marshal(val)
if err != nil {
return err
}
SetString(key, string(byteBuf))
return nil
}
func GetStringJson(key string, val interface{}) error {
ret, err := GetString(key)
if err != nil {
return err
}
err = json.Unmarshal([]byte(ret), val)
return err
}
func DelString(key string) error {
pubcacheservice := service.InstanceServiceMgr().FindService("CSyncCacheService")
if pubcacheservice == nil {
return errors.New("Cannot find CSyncCacheService")
}
pPubCacheService := pubcacheservice.(*CSyncCacheService)
syncCacheData := SyncCacheData{1, key, "", 0, pPubCacheService.nodeIdList[:], 0, 0}
for _, syncChan := range pPubCacheService.syncDataChanList {
pPubCacheService.tryPushSyncData(syncChan, &syncCacheData)
}
return nil
}
func SetString(key string, val string) error {
pubcacheservice := service.InstanceServiceMgr().FindService("CSyncCacheService")
if pubcacheservice == nil {
return errors.New("Cannot find CSyncCacheService")
}
//同步所有远程结点
pPubCacheService := pubcacheservice.(*CSyncCacheService)
syncCacheData := SyncCacheData{0, key, val, 0, pPubCacheService.nodeIdList[:], 0, 0}
for _, syncChan := range pPubCacheService.syncDataChanList {
pPubCacheService.tryPushSyncData(syncChan, &syncCacheData)
}
return nil
}
func GetString(key string) (string, error) {
pubcacheservice := service.InstanceServiceMgr().FindService("CSyncCacheService")
if pubcacheservice == nil {
return "", errors.New("Cannot find CSyncCacheService")
}
pPubCacheService := pubcacheservice.(*CSyncCacheService)
ret := pPubCacheService.mapCache.Get(key)
if ret == nil {
return "", errors.New(fmt.Sprintf("Cannot find key :%s", key))
}
return ret.(string), nil
}

137
sysservice/tcpservice.go Normal file
View File

@@ -0,0 +1,137 @@
package sysservice
import (
"fmt"
"github.com/duanhf2012/originnet/event"
"github.com/duanhf2012/originnet/log"
"github.com/duanhf2012/originnet/network"
"github.com/duanhf2012/originnet/service"
"sync"
)
type TcpService struct {
tcpServer network.TCPServer
service.Service
tcpService *TcpService
mapClientLocker sync.RWMutex
mapClient map[uint64] *Client
initClientId uint64
process network.Processor
}
const Default_MaxConnNum = 3000
const Default_PendingWriteNum = 10000
const Default_LittleEndian = false
const Default_MinMsgLen = 2
const Default_MaxMsgLen = 65535
func (slf *TcpService) OnInit() error{
iConfig := slf.GetServiceCfg()
if iConfig == nil {
return fmt.Errorf("%s service config is error!",slf.GetName())
}
tcpCfg := iConfig.(map[string]interface{})
addr,ok := tcpCfg["ListenAddr"]
if ok == false {
return fmt.Errorf("%s service config is error!",slf.GetName())
}
slf.tcpServer.Addr = addr.(string)
slf.tcpServer.MaxConnNum = Default_MaxConnNum
slf.tcpServer.PendingWriteNum = Default_PendingWriteNum
slf.tcpServer.LittleEndian = Default_LittleEndian
slf.tcpServer.MinMsgLen = Default_MinMsgLen
slf.tcpServer.MaxMsgLen = Default_MaxMsgLen
MaxConnNum,ok := tcpCfg["MaxConnNum"]
if ok == true {
slf.tcpServer.MaxConnNum = int(MaxConnNum.(float64))
}
PendingWriteNum,ok := tcpCfg["PendingWriteNum"]
if ok == true {
slf.tcpServer.PendingWriteNum = int(PendingWriteNum.(float64))
}
LittleEndian,ok := tcpCfg["LittleEndian"]
if ok == true {
slf.tcpServer.LittleEndian = LittleEndian.(bool)
}
MinMsgLen,ok := tcpCfg["MinMsgLen"]
if ok == true {
slf.tcpServer.MinMsgLen = uint32(MinMsgLen.(float64))
}
MaxMsgLen,ok := tcpCfg["MaxMsgLen"]
if ok == true {
slf.tcpServer.MaxMsgLen = uint32(MaxMsgLen.(float64))
}
slf.tcpServer.NewAgent =slf.NewClient
slf.tcpServer.Start()
//加载配置
return nil
}
func (slf *TcpService) SetProcessor(process network.Processor){
slf.process = process
}
func (slf *TcpService) NewClient(conn *network.TCPConn) network.Agent {
slf.mapClientLocker.Lock()
defer slf.mapClientLocker.Unlock()
for {
slf.initClientId+=1
_,ok := slf.mapClient[slf.initClientId]
if ok == true {
continue
}
pClient := &Client{tcpConn:conn}
slf.mapClient[slf.initClientId] = pClient
return pClient
}
return nil
}
type Client struct {
id uint64
tcpConn *network.TCPConn
tcpService *TcpService
}
type TcpPack struct {
ClientId uint64
Data interface{}
}
func (slf *Client) GetId() uint64 {
return slf.id
}
func (slf *Client) Run() {
slf.tcpService.GetEventReciver().NotifyEvent(&event.Event{Type:event.Sys_Event_Tcp_Connected,Data:&TcpPack{ClientId:slf.id}})
for{
bytes,err := slf.tcpConn.ReadMsg()
if err != nil {
log.Debug("read client id %d is error",err)
break
}
data,err:=slf.tcpService.process.Unmarshal(bytes)
if err != nil {
log.Debug("process.Unmarshal is error:%+v",err)
continue
}
slf.tcpService.GetEventReciver().NotifyEvent(&event.Event{Type:event.Sys_Event_Tcp_RecvPack,Data:&TcpPack{ClientId:slf.id,Data:data}})
}
}
func (slf *Client) OnClose(){
slf.tcpService.GetEventReciver().NotifyEvent(&event.Event{Type:event.Sys_Event_Tcp_DisConnected,Data:&TcpPack{ClientId:slf.id}})
slf.tcpService.mapClientLocker.Lock()
defer slf.tcpService.mapClientLocker.Unlock()
delete (slf.tcpService.mapClient,slf.GetId())
}

172
sysservice/tcpsocketpbservice.go
View File

@@ -1,172 +0,0 @@
package sysservice
import (
"errors"
"github.com/duanhf2012/origin/network"
"github.com/duanhf2012/origin/service"
"github.com/golang/protobuf/proto"
"reflect"
)
type TcpSocketPbService struct {
service.BaseService
listenaddr string
tcpsocketserver network.TcpSocketServer
MsgProcessor
}
type MessageHandler func(clientid uint64,msgtype uint16,msg proto.Message)
type MessageRecvHandler func(pClient *network.SClient,pPack *network.MsgBasePack)
type EventHandler func(clientid uint64)
type ExceptMsgHandler func(clientid uint64,pPack *network.MsgBasePack,err error)
type MsgProcessor struct {
mapMsg map[uint16]MessageInfo
connEvent EventHandler
disconnEvent EventHandler
exceptMsgHandler ExceptMsgHandler
messageRecvHandler MessageRecvHandler
}
func (slf *MsgProcessor) InitProcessor(){
slf.mapMsg = make(map[uint16]MessageInfo)
}
func (slf *MsgProcessor) RegMessage(msgtype uint16,msg proto.Message,handle MessageHandler){
var info MessageInfo
info.msgType = reflect.TypeOf(msg.(proto.Message))
info.msgHandler = handle
slf.mapMsg[msgtype] = info
}
func (slf *MsgProcessor) RegConnectEvent(eventHandler EventHandler){
slf.connEvent = eventHandler
}
func (slf *MsgProcessor) RegDisconnectEvent(eventHandler EventHandler){
slf.disconnEvent = eventHandler
}
func (slf *MsgProcessor) RegExceptMessage(exceptMsgHandler ExceptMsgHandler){
slf.exceptMsgHandler = exceptMsgHandler
}
func (slf *MsgProcessor) RegRecvMessage(msgHandler MessageRecvHandler){
slf.messageRecvHandler = msgHandler
}
func (slf *MsgProcessor) OnExceptMsg (pClient *network.SClient,pPack *network.MsgBasePack,err error){
if slf.exceptMsgHandler!=nil {
slf.exceptMsgHandler(pClient.GetId(),pPack,err)
}else{
pClient.Close()
//记录日志
service.GetLogger().Printf(service.LEVER_WARN, "OnExceptMsg packtype %d,error %+v",pPack.PackType,err)
}
}
func NewTcpSocketPbService(listenaddr string) *TcpSocketPbService {
ts := new(TcpSocketPbService)
ts.listenaddr = listenaddr
ts.mapMsg = make(map[uint16]MessageInfo,1)
ts.tcpsocketserver.Register(listenaddr,ts)
return ts
}
func (slf *TcpSocketPbService) OnInit() error {
return nil
}
func (slf *TcpSocketPbService) OnRun() bool {
slf.tcpsocketserver.Start()
return false
}
type MessageInfo struct {
msgType reflect.Type
msgHandler MessageHandler
}
func (slf *TcpSocketPbService) RegMessage(msgtype uint16,msg proto.Message,handle MessageHandler){
var info MessageInfo
info.msgType = reflect.TypeOf(msg.(proto.Message))
info.msgHandler = handle
slf.mapMsg[msgtype] = info
}
func (slf *TcpSocketPbService) OnConnected(pClient *network.SClient){
if slf.connEvent!=nil {
slf.connEvent(pClient.GetId())
}
}
func (slf *TcpSocketPbService) OnDisconnect(pClient *network.SClient){
if slf.disconnEvent!=nil {
slf.disconnEvent(pClient.GetId())
}
}
func (slf *MsgProcessor) Handle(pClient *network.SClient,pPack *network.MsgBasePack){
if info, ok := slf.mapMsg[pPack.PackType]; ok {
msg := reflect.New(info.msgType.Elem()).Interface()
tmp := msg.(proto.Message)
err := proto.Unmarshal(pPack.Body, tmp)
if err != nil {
slf.OnExceptMsg(pClient,pPack,err)
return
}
info.msgHandler(pClient.GetId(),pPack.PackType, msg.(proto.Message))
return
}else if slf.messageRecvHandler!=nil {
slf.messageRecvHandler(pClient,pPack)
return
}
slf.OnExceptMsg(pClient,pPack,errors.New("not found PackType"))
}
func (slf *TcpSocketPbService) OnRecvMsg(pClient *network.SClient, pPack *network.MsgBasePack){
slf.Handle(pClient,pPack)
}
func DefaultTSPbService() *TcpSocketPbService{
iservice := service.InstanceServiceMgr().FindService("TcpSocketPbService")
if iservice == nil {
return nil
}
return iservice.(*TcpSocketPbService)
}
func GetTcpSocketPbService(serviceName string) *TcpSocketPbService{
iservice := service.InstanceServiceMgr().FindService(serviceName)
if iservice == nil {
return nil
}
return iservice.(*TcpSocketPbService)
}
func (slf *TcpSocketPbService) SendMsg(clientid uint64,packtype uint16,message proto.Message) error{
return slf.tcpsocketserver.SendMsg(clientid,packtype,message)
}
func (slf *TcpSocketPbService) Close(clientid uint64) error{
return slf.tcpsocketserver.Close(clientid)
}
func (slf *TcpSocketPbService) Send(clientid uint64,pack *network.MsgBasePack) error {
return slf.tcpsocketserver.Send(clientid,pack)
}

41
sysservice/wsagentservice.go
View File

@@ -1,41 +0,0 @@
package sysservice
import (
"github.com/duanhf2012/origin/network"
"github.com/duanhf2012/origin/service"
)
type WSAgentService struct {
service.BaseService
agentserver network.WSAgentServer
pattern string
port uint16
bEnableCompression bool
}
func (ws *WSAgentService) OnInit() error {
ws.AddModule(&ws.agentserver)
ws.agentserver.Init(ws.port)
return nil
}
func (ws *WSAgentService) OnRun() bool {
ws.agentserver.Start()
return false
}
func NewWSAgentService(port uint16) *WSAgentService {
wss := new(WSAgentService)
wss.port = port
return wss
}
func (ws *WSAgentService) OnDestory() error {
return nil
}
func (ws *WSAgentService) SetupAgent(pattern string, agent network.IAgent, bEnableCompression bool) {
ws.agentserver.SetupAgent(pattern, agent, bEnableCompression)
}

47
sysservice/wsserverservice.go
View File

@@ -1,47 +0,0 @@
package sysservice
import (
"github.com/duanhf2012/origin/network"
"github.com/duanhf2012/origin/service"
)
type WSServerService struct {
service.BaseService
wsserver network.WebsocketServer
pattern string
port uint16
messageReciver network.IMessageReceiver
bEnableCompression bool
}
func (ws *WSServerService) OnInit() error {
ws.wsserver.Init(ws.port)
return nil
}
func (ws *WSServerService) OnRun() bool {
ws.wsserver.Start()
return false
}
func NewWSServerService(port uint16) *WSServerService {
wss := new(WSServerService)
wss.port = port
return wss
}
func (ws *WSServerService) OnDestory() error {
return nil
}
func (ws *WSServerService) SetupReciver(pattern string, messageReciver network.IMessageReceiver, bEnableCompression bool) {
ws.wsserver.SetupReciver(pattern, messageReciver, bEnableCompression)
}
func (slf *WSServerService) SetWSS(certfile string, keyfile string) bool {
slf.wsserver.SetWSS(certfile, keyfile)
return true
}

4
sysservice/wsservice.go Normal file
View File

@@ -0,0 +1,4 @@
package sysservice

10
util/Log.go
View File

@@ -1,10 +0,0 @@
package util
type PrintLog func(uint, string, ...interface{})
var Log PrintLog

94
util/Timer.go
View File

@@ -1,94 +0,0 @@
package util
import "time"
type Timer struct {
lasttime int64
timeinterval int64
setupZeroDBase time.Duration //0表示普通模式 1表示切换分钟模式
}
func (slf *Timer) GetTimerinterval() int64 {
return slf.timeinterval
}
func (slf *Timer) SetupTimer(ms int32) {
slf.lasttime = time.Now().UnixNano()
slf.timeinterval = int64(ms) * 1e6
}
func (slf *Timer) SetupTimerEx(tm time.Duration) {
slf.lasttime = time.Now().UnixNano()
slf.timeinterval = int64(tm)
}
func (slf *Timer) SetupTimerDouble() {
slf.lasttime = time.Now().UnixNano()
slf.timeinterval *= 2
}
func (slf *Timer) SetTimerHalf() {
slf.lasttime = time.Now().UnixNano()
slf.timeinterval /= 2
}
//检查整点分钟数触发
func (slf *Timer) SetupZeroTimer(baseD time.Duration, interval int64) {
timeNow := time.Now()
nt := timeNow.Truncate(baseD)
slf.lasttime = nt.UnixNano()
slf.timeinterval = baseD.Nanoseconds() * interval
slf.setupZeroDBase = baseD
}
func (slf *Timer) ResetStartTime() {
slf.lasttime = 0
}
func (slf *Timer) CheckTimeOut() bool {
now := time.Now()
if slf.setupZeroDBase.Nanoseconds() == 0 {
if now.UnixNano() > slf.lasttime+slf.timeinterval {
slf.lasttime = now.UnixNano()
return true
}
} else { //整点模式
if now.UnixNano() > slf.lasttime+slf.timeinterval {
slf.SetupZeroTimer(slf.setupZeroDBase, slf.timeinterval/slf.setupZeroDBase.Nanoseconds())
return true
}
}
return false
}
func GetTomorrowTimestamp() int64{
timeStr := time.Now().Format("2006-01-02")
t, _ := time.ParseInLocation("2006-01-02 15:04:05", timeStr+" 23:59:59", time.Local)
return t.Unix()+1
}
func IsSameDay(timeFir, timeSec int64) bool{
firTime:=time.Unix(timeFir,0)
secTime:=time.Unix(timeSec,0)
if firTime.Day()==secTime.Day()&&firTime.Month()==secTime.Month()&&firTime.Year()==secTime.Year(){
return true
}
return false
}
func IsCrossOneDay(timeFir,timeSec int64)bool{
firTime := time.Unix(timeFir,0)
secTime := time.Unix(timeSec,0)
firTime.Add(time.Hour*24).Day()
if firTime.Day() == secTime.Day() &&
firTime.Month()==secTime.Month()&&
firTime.Year()==secTime.Year(){
return true
}
return false
}

17
util/aes_encrypt.go → util/aesencrypt/aes_encrypt.go
View File

@@ -1,17 +1,4 @@
// Copyright 2014 mqantserver Author. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package util
package aesencrypt
import (
"crypto/aes"
@@ -86,4 +73,4 @@ func (this *AesEncrypt) Decrypt(src []byte) (strDesc string, err error) {
aesDecrypter := cipher.NewCFBDecrypter(aesBlockDecrypter, iv)
aesDecrypter.XORKeyStream(decrypted, src)
return string(decrypted), nil
}
}

8
util/Coroutine.go → util/coroutine/coroutine.go
View File

@@ -1,4 +1,4 @@
package util
package coroutine
import (
"fmt"
@@ -12,11 +12,7 @@ func F(callback interface{},recoverNum int, args ...interface{}) {
var coreInfo string
coreInfo = string(debug.Stack())
coreInfo += "\n" + fmt.Sprintf("Core information is %v\n", r)
if Log != nil {
Log(5, coreInfo)
} else {
fmt.Print(coreInfo)
}
fmt.Print(coreInfo)
if recoverNum==-1 ||recoverNum-1 >= 0 {
recoverNum -= 1

7
util/deepcopy.go → util/deepcopy/deepcopy.go
View File

@@ -1,9 +1,6 @@
package util
package deepcopy
// reference: https://github.com/mohae/deepcopy
import (
"reflect"
)
import "reflect"
func deepCopy(dst, src reflect.Value) {
switch src.Kind() {

35
util/example_test.go
View File

@@ -1,35 +0,0 @@
package util_test
import (
"fmt"
"testing"
"github.com/duanhf2012/origin/util"
)
func ExampleMap(t *testing.T) {
maps := util.NewMapEx()
for i := 0; i < 10000; i++ {
maps.Set(i, i)
}
for i := 0; i < 10000; i++ {
ret := maps.Get(i)
if ret.(int) != i {
fmt.Printf("cannot find i:%d\n", i)
}
}
for i := 0; i < 10000; i++ {
maps.LockSet(i, func(key interface{}, val interface{}) interface{} {
return val.(int) + 1
})
}
for i := 0; i < 10000; i++ {
ret := maps.Get(i)
if ret.(int) != i {
fmt.Printf("cannot find i:%d\n", i)
}
}
}

1
util/has/hash.go Normal file
View File

@@ -0,0 +1 @@
package has

2
util/hash/hash.go
View File

@@ -4,4 +4,4 @@ import "hash/crc32"
func HashNumber(s string) uint {
return uint(crc32.ChecksumIEEE([]byte(s)))
}
}

101
util/queue.go
View File

@@ -1,101 +0,0 @@
/*
Package queue provides a fast, ring-buffer queue based on the version suggested by Dariusz Górecki.
Using this instead of other, simpler, queue implementations (slice+append or linked list) provides
substantial memory and time benefits, and fewer GC pauses.
The queue implemented here is as fast as it is for an additional reason: it is *not* thread-safe.
*/
package util
// minQueueLen is smallest capacity that queue may have.
// Must be power of 2 for bitwise modulus: x % n == x & (n - 1).
const minQueueLen = 16
// Queue represents a single instance of the queue data structure.
type Queue struct {
buf []interface{}
head, tail, count int
}
// New constructs and returns a new Queue.
func NewQueue() *Queue {
return &Queue{
buf: make([]interface{}, minQueueLen),
}
}
// Length returns the number of elements currently stored in the queue.
func (q *Queue) Length() int {
return q.count
}
// resizes the queue to fit exactly twice its current contents
// this can result in shrinking if the queue is less than half-full
func (q *Queue) resize() {
newBuf := make([]interface{}, q.count<<1)
if q.tail > q.head {
copy(newBuf, q.buf[q.head:q.tail])
} else {
n := copy(newBuf, q.buf[q.head:])
copy(newBuf[n:], q.buf[:q.tail])
}
q.head = 0
q.tail = q.count
q.buf = newBuf
}
// Add puts an element on the end of the queue.
func (q *Queue) Add(elem interface{}) {
if q.count == len(q.buf) {
q.resize()
}
q.buf[q.tail] = elem
// bitwise modulus
q.tail = (q.tail + 1) & (len(q.buf) - 1)
q.count++
}
// Peek returns the element at the head of the queue. This call panics
// if the queue is empty.
func (q *Queue) Peek() interface{} {
if q.count <= 0 {
panic("queue: Peek() called on empty queue")
}
return q.buf[q.head]
}
// Get returns the element at index i in the queue. If the index is
// invalid, the call will panic. This method accepts both positive and
// negative index values. Index 0 refers to the first element, and
// index -1 refers to the last.
func (q *Queue) Get(i int) interface{} {
// If indexing backwards, convert to positive index.
if i < 0 {
i += q.count
}
if i < 0 || i >= q.count {
panic("queue: Get() called with index out of range")
}
// bitwise modulus
return q.buf[(q.head+i)&(len(q.buf)-1)]
}
// Remove removes and returns the element from the front of the queue. If the
// queue is empty, the call will panic.
func (q *Queue) Remove() interface{} {
if q.count <= 0 {
panic("queue: Remove() called on empty queue")
}
ret := q.buf[q.head]
q.buf[q.head] = nil
// bitwise modulus
q.head = (q.head + 1) & (len(q.buf) - 1)
q.count--
// Resize down if buffer 1/4 full.
if len(q.buf) > minQueueLen && (q.count<<2) == len(q.buf) {
q.resize()
}
return ret
}

23
util/queue/.gitignore vendored
View File

@@ -1,23 +0,0 @@
# Compiled Object files, Static and Dynamic libs (Shared Objects)
*.o
*.a
*.so
# Folders
_obj
_test
# Architecture specific extensions/prefixes
*.[568vq]
[568vq].out
*.cgo1.go
*.cgo2.c
_cgo_defun.c
_cgo_gotypes.go
_cgo_export.*
_testmain.go
*.exe
*.test

7
util/queue/.travis.yml
View File

@@ -1,7 +0,0 @@
language: go
sudo: false
go:
- 1.2
- 1.3
- 1.4

Some files were not shown because too many files have changed in this diff Show More