--- /dev/null
+= CodeRay - Trunk folder structure\r
+\r
+== bench - Benchmarking system\r
+\r
+All benchmarking stuff goes here.\r
+\r
+Test inputs are stored in files named <code>example.<lang></code>.\r
+Test outputs go to <code>bench/test.<encoder-default-file-extension></code>.\r
+\r
+Run <code>bench/bench.rb</code> to get a usage description.\r
+\r
+Run <code>rake bench</code> to perform an example benchmark.\r
+\r
+\r
+== bin - Scripts\r
+\r
+Executional files for CodeRay.\r
+\r
+\r
+== demo - Demos and functional tests\r
+\r
+Demonstrational scripts to show of CodeRay's features.\r
+\r
+Run them as functional tests with <code>rake test:demos</code>.\r
+\r
+\r
+== etc - Lots of stuff\r
+\r
+Some addidtional files for CodeRay, mainly graphics and Vim scripts.\r
+\r
+\r
+== gem_server - Gem output folder\r
+\r
+For <code>rake gem</code>.\r
+\r
+\r
+== lib - CodeRay library code\r
+\r
+This is the base directory for the CodeRay library.\r
+\r
+\r
+== rake_helpers - Rake helper libraries\r
+\r
+Some files to enhance Rake, including the Autumnal Rdoc template and some scripts.\r
+\r
+\r
+== test - Tests\r
+\r
+Tests for the scanners.\r
+\r
+Each language has its own subfolder and sub-suite.\r
+\r
+Run with <code>rake test</code>.\r
--- /dev/null
+ GNU LESSER GENERAL PUBLIC LICENSE
+ Version 2.1, February 1999
+
+ Copyright (C) 1991, 1999 Free Software Foundation, Inc.
+ 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+[This is the first released version of the Lesser GPL. It also counts
+ as the successor of the GNU Library Public License, version 2, hence
+ the version number 2.1.]
+
+ Preamble
+
+ The licenses for most software are designed to take away your
+freedom to share and change it. By contrast, the GNU General Public
+Licenses are intended to guarantee your freedom to share and change
+free software--to make sure the software is free for all its users.
+
+ This license, the Lesser General Public License, applies to some
+specially designated software packages--typically libraries--of the
+Free Software Foundation and other authors who decide to use it. You
+can use it too, but we suggest you first think carefully about whether
+this license or the ordinary General Public License is the better
+strategy to use in any particular case, based on the explanations below.
+
+ When we speak of free software, we are referring to freedom of use,
+not price. Our General Public Licenses are designed to make sure that
+you have the freedom to distribute copies of free software (and charge
+for this service if you wish); that you receive source code or can get
+it if you want it; that you can change the software and use pieces of
+it in new free programs; and that you are informed that you can do
+these things.
+
+ To protect your rights, we need to make restrictions that forbid
+distributors to deny you these rights or to ask you to surrender these
+rights. These restrictions translate to certain responsibilities for
+you if you distribute copies of the library or if you modify it.
+
+ For example, if you distribute copies of the library, whether gratis
+or for a fee, you must give the recipients all the rights that we gave
+you. You must make sure that they, too, receive or can get the source
+code. If you link other code with the library, you must provide
+complete object files to the recipients, so that they can relink them
+with the library after making changes to the library and recompiling
+it. And you must show them these terms so they know their rights.
+
+ We protect your rights with a two-step method: (1) we copyright the
+library, and (2) we offer you this license, which gives you legal
+permission to copy, distribute and/or modify the library.
+
+ To protect each distributor, we want to make it very clear that
+there is no warranty for the free library. Also, if the library is
+modified by someone else and passed on, the recipients should know
+that what they have is not the original version, so that the original
+author's reputation will not be affected by problems that might be
+introduced by others.
+\f
+ Finally, software patents pose a constant threat to the existence of
+any free program. We wish to make sure that a company cannot
+effectively restrict the users of a free program by obtaining a
+restrictive license from a patent holder. Therefore, we insist that
+any patent license obtained for a version of the library must be
+consistent with the full freedom of use specified in this license.
+
+ Most GNU software, including some libraries, is covered by the
+ordinary GNU General Public License. This license, the GNU Lesser
+General Public License, applies to certain designated libraries, and
+is quite different from the ordinary General Public License. We use
+this license for certain libraries in order to permit linking those
+libraries into non-free programs.
+
+ When a program is linked with a library, whether statically or using
+a shared library, the combination of the two is legally speaking a
+combined work, a derivative of the original library. The ordinary
+General Public License therefore permits such linking only if the
+entire combination fits its criteria of freedom. The Lesser General
+Public License permits more lax criteria for linking other code with
+the library.
+
+ We call this license the "Lesser" General Public License because it
+does Less to protect the user's freedom than the ordinary General
+Public License. It also provides other free software developers Less
+of an advantage over competing non-free programs. These disadvantages
+are the reason we use the ordinary General Public License for many
+libraries. However, the Lesser license provides advantages in certain
+special circumstances.
+
+ For example, on rare occasions, there may be a special need to
+encourage the widest possible use of a certain library, so that it becomes
+a de-facto standard. To achieve this, non-free programs must be
+allowed to use the library. A more frequent case is that a free
+library does the same job as widely used non-free libraries. In this
+case, there is little to gain by limiting the free library to free
+software only, so we use the Lesser General Public License.
+
+ In other cases, permission to use a particular library in non-free
+programs enables a greater number of people to use a large body of
+free software. For example, permission to use the GNU C Library in
+non-free programs enables many more people to use the whole GNU
+operating system, as well as its variant, the GNU/Linux operating
+system.
+
+ Although the Lesser General Public License is Less protective of the
+users' freedom, it does ensure that the user of a program that is
+linked with the Library has the freedom and the wherewithal to run
+that program using a modified version of the Library.
+
+ The precise terms and conditions for copying, distribution and
+modification follow. Pay close attention to the difference between a
+"work based on the library" and a "work that uses the library". The
+former contains code derived from the library, whereas the latter must
+be combined with the library in order to run.
+\f
+ GNU LESSER GENERAL PUBLIC LICENSE
+ TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+
+ 0. This License Agreement applies to any software library or other
+program which contains a notice placed by the copyright holder or
+other authorized party saying it may be distributed under the terms of
+this Lesser General Public License (also called "this License").
+Each licensee is addressed as "you".
+
+ A "library" means a collection of software functions and/or data
+prepared so as to be conveniently linked with application programs
+(which use some of those functions and data) to form executables.
+
+ The "Library", below, refers to any such software library or work
+which has been distributed under these terms. A "work based on the
+Library" means either the Library or any derivative work under
+copyright law: that is to say, a work containing the Library or a
+portion of it, either verbatim or with modifications and/or translated
+straightforwardly into another language. (Hereinafter, translation is
+included without limitation in the term "modification".)
+
+ "Source code" for a work means the preferred form of the work for
+making modifications to it. For a library, complete source code means
+all the source code for all modules it contains, plus any associated
+interface definition files, plus the scripts used to control compilation
+and installation of the library.
+
+ Activities other than copying, distribution and modification are not
+covered by this License; they are outside its scope. The act of
+running a program using the Library is not restricted, and output from
+such a program is covered only if its contents constitute a work based
+on the Library (independent of the use of the Library in a tool for
+writing it). Whether that is true depends on what the Library does
+and what the program that uses the Library does.
+
+ 1. You may copy and distribute verbatim copies of the Library's
+complete source code as you receive it, in any medium, provided that
+you conspicuously and appropriately publish on each copy an
+appropriate copyright notice and disclaimer of warranty; keep intact
+all the notices that refer to this License and to the absence of any
+warranty; and distribute a copy of this License along with the
+Library.
+
+ You may charge a fee for the physical act of transferring a copy,
+and you may at your option offer warranty protection in exchange for a
+fee.
+\f
+ 2. You may modify your copy or copies of the Library or any portion
+of it, thus forming a work based on the Library, and copy and
+distribute such modifications or work under the terms of Section 1
+above, provided that you also meet all of these conditions:
+
+ a) The modified work must itself be a software library.
+
+ b) You must cause the files modified to carry prominent notices
+ stating that you changed the files and the date of any change.
+
+ c) You must cause the whole of the work to be licensed at no
+ charge to all third parties under the terms of this License.
+
+ d) If a facility in the modified Library refers to a function or a
+ table of data to be supplied by an application program that uses
+ the facility, other than as an argument passed when the facility
+ is invoked, then you must make a good faith effort to ensure that,
+ in the event an application does not supply such function or
+ table, the facility still operates, and performs whatever part of
+ its purpose remains meaningful.
+
+ (For example, a function in a library to compute square roots has
+ a purpose that is entirely well-defined independent of the
+ application. Therefore, Subsection 2d requires that any
+ application-supplied function or table used by this function must
+ be optional: if the application does not supply it, the square
+ root function must still compute square roots.)
+
+These requirements apply to the modified work as a whole. If
+identifiable sections of that work are not derived from the Library,
+and can be reasonably considered independent and separate works in
+themselves, then this License, and its terms, do not apply to those
+sections when you distribute them as separate works. But when you
+distribute the same sections as part of a whole which is a work based
+on the Library, the distribution of the whole must be on the terms of
+this License, whose permissions for other licensees extend to the
+entire whole, and thus to each and every part regardless of who wrote
+it.
+
+Thus, it is not the intent of this section to claim rights or contest
+your rights to work written entirely by you; rather, the intent is to
+exercise the right to control the distribution of derivative or
+collective works based on the Library.
+
+In addition, mere aggregation of another work not based on the Library
+with the Library (or with a work based on the Library) on a volume of
+a storage or distribution medium does not bring the other work under
+the scope of this License.
+
+ 3. You may opt to apply the terms of the ordinary GNU General Public
+License instead of this License to a given copy of the Library. To do
+this, you must alter all the notices that refer to this License, so
+that they refer to the ordinary GNU General Public License, version 2,
+instead of to this License. (If a newer version than version 2 of the
+ordinary GNU General Public License has appeared, then you can specify
+that version instead if you wish.) Do not make any other change in
+these notices.
+\f
+ Once this change is made in a given copy, it is irreversible for
+that copy, so the ordinary GNU General Public License applies to all
+subsequent copies and derivative works made from that copy.
+
+ This option is useful when you wish to copy part of the code of
+the Library into a program that is not a library.
+
+ 4. You may copy and distribute the Library (or a portion or
+derivative of it, under Section 2) in object code or executable form
+under the terms of Sections 1 and 2 above provided that you accompany
+it with the complete corresponding machine-readable source code, which
+must be distributed under the terms of Sections 1 and 2 above on a
+medium customarily used for software interchange.
+
+ If distribution of object code is made by offering access to copy
+from a designated place, then offering equivalent access to copy the
+source code from the same place satisfies the requirement to
+distribute the source code, even though third parties are not
+compelled to copy the source along with the object code.
+
+ 5. A program that contains no derivative of any portion of the
+Library, but is designed to work with the Library by being compiled or
+linked with it, is called a "work that uses the Library". Such a
+work, in isolation, is not a derivative work of the Library, and
+therefore falls outside the scope of this License.
+
+ However, linking a "work that uses the Library" with the Library
+creates an executable that is a derivative of the Library (because it
+contains portions of the Library), rather than a "work that uses the
+library". The executable is therefore covered by this License.
+Section 6 states terms for distribution of such executables.
+
+ When a "work that uses the Library" uses material from a header file
+that is part of the Library, the object code for the work may be a
+derivative work of the Library even though the source code is not.
+Whether this is true is especially significant if the work can be
+linked without the Library, or if the work is itself a library. The
+threshold for this to be true is not precisely defined by law.
+
+ If such an object file uses only numerical parameters, data
+structure layouts and accessors, and small macros and small inline
+functions (ten lines or less in length), then the use of the object
+file is unrestricted, regardless of whether it is legally a derivative
+work. (Executables containing this object code plus portions of the
+Library will still fall under Section 6.)
+
+ Otherwise, if the work is a derivative of the Library, you may
+distribute the object code for the work under the terms of Section 6.
+Any executables containing that work also fall under Section 6,
+whether or not they are linked directly with the Library itself.
+\f
+ 6. As an exception to the Sections above, you may also combine or
+link a "work that uses the Library" with the Library to produce a
+work containing portions of the Library, and distribute that work
+under terms of your choice, provided that the terms permit
+modification of the work for the customer's own use and reverse
+engineering for debugging such modifications.
+
+ You must give prominent notice with each copy of the work that the
+Library is used in it and that the Library and its use are covered by
+this License. You must supply a copy of this License. If the work
+during execution displays copyright notices, you must include the
+copyright notice for the Library among them, as well as a reference
+directing the user to the copy of this License. Also, you must do one
+of these things:
+
+ a) Accompany the work with the complete corresponding
+ machine-readable source code for the Library including whatever
+ changes were used in the work (which must be distributed under
+ Sections 1 and 2 above); and, if the work is an executable linked
+ with the Library, with the complete machine-readable "work that
+ uses the Library", as object code and/or source code, so that the
+ user can modify the Library and then relink to produce a modified
+ executable containing the modified Library. (It is understood
+ that the user who changes the contents of definitions files in the
+ Library will not necessarily be able to recompile the application
+ to use the modified definitions.)
+
+ b) Use a suitable shared library mechanism for linking with the
+ Library. A suitable mechanism is one that (1) uses at run time a
+ copy of the library already present on the user's computer system,
+ rather than copying library functions into the executable, and (2)
+ will operate properly with a modified version of the library, if
+ the user installs one, as long as the modified version is
+ interface-compatible with the version that the work was made with.
+
+ c) Accompany the work with a written offer, valid for at
+ least three years, to give the same user the materials
+ specified in Subsection 6a, above, for a charge no more
+ than the cost of performing this distribution.
+
+ d) If distribution of the work is made by offering access to copy
+ from a designated place, offer equivalent access to copy the above
+ specified materials from the same place.
+
+ e) Verify that the user has already received a copy of these
+ materials or that you have already sent this user a copy.
+
+ For an executable, the required form of the "work that uses the
+Library" must include any data and utility programs needed for
+reproducing the executable from it. However, as a special exception,
+the materials to be distributed need not include anything that is
+normally distributed (in either source or binary form) with the major
+components (compiler, kernel, and so on) of the operating system on
+which the executable runs, unless that component itself accompanies
+the executable.
+
+ It may happen that this requirement contradicts the license
+restrictions of other proprietary libraries that do not normally
+accompany the operating system. Such a contradiction means you cannot
+use both them and the Library together in an executable that you
+distribute.
+\f
+ 7. You may place library facilities that are a work based on the
+Library side-by-side in a single library together with other library
+facilities not covered by this License, and distribute such a combined
+library, provided that the separate distribution of the work based on
+the Library and of the other library facilities is otherwise
+permitted, and provided that you do these two things:
+
+ a) Accompany the combined library with a copy of the same work
+ based on the Library, uncombined with any other library
+ facilities. This must be distributed under the terms of the
+ Sections above.
+
+ b) Give prominent notice with the combined library of the fact
+ that part of it is a work based on the Library, and explaining
+ where to find the accompanying uncombined form of the same work.
+
+ 8. You may not copy, modify, sublicense, link with, or distribute
+the Library except as expressly provided under this License. Any
+attempt otherwise to copy, modify, sublicense, link with, or
+distribute the Library is void, and will automatically terminate your
+rights under this License. However, parties who have received copies,
+or rights, from you under this License will not have their licenses
+terminated so long as such parties remain in full compliance.
+
+ 9. You are not required to accept this License, since you have not
+signed it. However, nothing else grants you permission to modify or
+distribute the Library or its derivative works. These actions are
+prohibited by law if you do not accept this License. Therefore, by
+modifying or distributing the Library (or any work based on the
+Library), you indicate your acceptance of this License to do so, and
+all its terms and conditions for copying, distributing or modifying
+the Library or works based on it.
+
+ 10. Each time you redistribute the Library (or any work based on the
+Library), the recipient automatically receives a license from the
+original licensor to copy, distribute, link with or modify the Library
+subject to these terms and conditions. You may not impose any further
+restrictions on the recipients' exercise of the rights granted herein.
+You are not responsible for enforcing compliance by third parties with
+this License.
+\f
+ 11. If, as a consequence of a court judgment or allegation of patent
+infringement or for any other reason (not limited to patent issues),
+conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License. If you cannot
+distribute so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you
+may not distribute the Library at all. For example, if a patent
+license would not permit royalty-free redistribution of the Library by
+all those who receive copies directly or indirectly through you, then
+the only way you could satisfy both it and this License would be to
+refrain entirely from distribution of the Library.
+
+If any portion of this section is held invalid or unenforceable under any
+particular circumstance, the balance of the section is intended to apply,
+and the section as a whole is intended to apply in other circumstances.
+
+It is not the purpose of this section to induce you to infringe any
+patents or other property right claims or to contest validity of any
+such claims; this section has the sole purpose of protecting the
+integrity of the free software distribution system which is
+implemented by public license practices. Many people have made
+generous contributions to the wide range of software distributed
+through that system in reliance on consistent application of that
+system; it is up to the author/donor to decide if he or she is willing
+to distribute software through any other system and a licensee cannot
+impose that choice.
+
+This section is intended to make thoroughly clear what is believed to
+be a consequence of the rest of this License.
+
+ 12. If the distribution and/or use of the Library is restricted in
+certain countries either by patents or by copyrighted interfaces, the
+original copyright holder who places the Library under this License may add
+an explicit geographical distribution limitation excluding those countries,
+so that distribution is permitted only in or among countries not thus
+excluded. In such case, this License incorporates the limitation as if
+written in the body of this License.
+
+ 13. The Free Software Foundation may publish revised and/or new
+versions of the Lesser General Public License from time to time.
+Such new versions will be similar in spirit to the present version,
+but may differ in detail to address new problems or concerns.
+
+Each version is given a distinguishing version number. If the Library
+specifies a version number of this License which applies to it and
+"any later version", you have the option of following the terms and
+conditions either of that version or of any later version published by
+the Free Software Foundation. If the Library does not specify a
+license version number, you may choose any version ever published by
+the Free Software Foundation.
+\f
+ 14. If you wish to incorporate parts of the Library into other free
+programs whose distribution conditions are incompatible with these,
+write to the author to ask for permission. For software which is
+copyrighted by the Free Software Foundation, write to the Free
+Software Foundation; we sometimes make exceptions for this. Our
+decision will be guided by the two goals of preserving the free status
+of all derivatives of our free software and of promoting the sharing
+and reuse of software generally.
+
+ NO WARRANTY
+
+ 15. BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO
+WARRANTY FOR THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW.
+EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR
+OTHER PARTIES PROVIDE THE LIBRARY "AS IS" WITHOUT WARRANTY OF ANY
+KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE
+LIBRARY IS WITH YOU. SHOULD THE LIBRARY PROVE DEFECTIVE, YOU ASSUME
+THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
+
+ 16. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN
+WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY
+AND/OR REDISTRIBUTE THE LIBRARY AS PERMITTED ABOVE, BE LIABLE TO YOU
+FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR
+CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE
+LIBRARY (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING
+RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A
+FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF
+SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH
+DAMAGES.
+
+ END OF TERMS AND CONDITIONS
+\f
+ How to Apply These Terms to Your New Libraries
+
+ If you develop a new library, and you want it to be of the greatest
+possible use to the public, we recommend making it free software that
+everyone can redistribute and change. You can do so by permitting
+redistribution under these terms (or, alternatively, under the terms of the
+ordinary General Public License).
+
+ To apply these terms, attach the following notices to the library. It is
+safest to attach them to the start of each source file to most effectively
+convey the exclusion of warranty; and each file should have at least the
+"copyright" line and a pointer to where the full notice is found.
+
+ <one line to give the library's name and a brief idea of what it does.>
+ Copyright (C) <year> <name of author>
+
+ This library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ This library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with this library; if not, write to the Free Software
+ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+
+Also add information on how to contact you by electronic and paper mail.
+
+You should also get your employer (if you work as a programmer) or your
+school, if any, to sign a "copyright disclaimer" for the library, if
+necessary. Here is a sample; alter the names:
+
+ Yoyodyne, Inc., hereby disclaims all copyright interest in the
+ library `Frob' (a library for tweaking knobs) written by James Random Hacker.
+
+ <signature of Ty Coon>, 1 April 1990
+ Ty Coon, President of Vice
+
+That's all there is to it!
+
+
= CodeRay
[- Tired of blue'n'gray? Try the original version of this documentation on
-coderay.rubychan.de[http://coderay.rubychan.de/doc/] (use Ctrl+Click to open it in its own frame.) -]
+http://rd.cYcnus.de/coderay/doc (use Ctrl+Click to open it in its own frame.) -]
== About
CodeRay is a Ruby library for syntax highlighting.
* is what everybody should have on their website
* solves all your problems and makes the girls run after you
-Version: 0.9.0
+Version: 0.7.4 (2006.october.20)
Author:: murphy (Kornelius Kalnbach)
Contact:: murphy rubychan de
Website:: coderay.rubychan.de[http://coderay.rubychan.de]
License:: GNU LGPL; see LICENSE file in the main directory.
+Subversion:: $Id: README 219 2006-10-20 15:52:25Z murphy $
+
+-----
== Installation
% gem install coderay
+Since CodeRay is still in beta stage, nightly buildy may be useful:
+
+ % gem install coderay -rs rd.cYcnus.de/coderay
+
=== Dependencies
-CodeRay needs Ruby 1.8.6 or later. It also runs with Ruby 1.9.1+ and JRuby 1.1+.
+CodeRay needs Ruby 1.8 and the
+strscan[http://www.ruby-doc.org/stdlib/libdoc/strscan/rdoc/index.htm]
+library (part of the standard library.) It should also run with Ruby 1.9 and
+yarv.
== Example Usage
See CodeRay.
-Please report errors in this documentation to <murphy rubychan de>.
+Please report errors in this documentation to <coderay cycnus de>.
+-----
+
== Credits
=== Special Thanks to
* licenser (Heinz N. Gies) for ending my QBasic career, inventing the Coder
project and the input/output plugin system.
CodeRay would not exist without him.
-* bovi (Daniel Bovensiepen) for helping me out on various occasions.
=== Thanks to
* Caleb Clausen for writing RubyLexer (see
http://rubyforge.org/projects/rubylexer) and lots of very interesting mail
traffic
-* birkenfeld (Georg Brandl) and mitsuhiku (Arnim Ronacher) for PyKleur, now pygments.
- You guys rock!
+* birkenfeld (Georg Brandl) and mitsuhiku (Arnim Ronacher) for PyKleur. You
+ guys rock!
* Jamis Buck for writing Syntax (see http://rubyforge.org/projects/syntax)
I got some useful ideas from it.
* Doug Kearns and everyone else who worked on ruby.vim - it not only helped me
coding CodeRay, but also gave me a wonderful target to reach for the Ruby
scanner.
-* everyone who uses CodeBB on http://www.rubyforen.de and http://www.python-forum.de
-* iGEL, magichisoka, manveru, WoNáDo and everyone I forgot from rubyforen.de
-* Dethix from ruby-mine.de
-* zickzackw
-* Dookie (who is no longer with us...) and Leonidas from http://www.python-forum.de
+* everyone who used CodeBB on http://www.rubyforen.de and
+ http://www.infhu.de/mx
+* iGEL, magichisoka, manveru, WoNáDo and everyone I forgot from rubyforen.de
+* Daniel and Dethix from ruby-mine.de
+* Dookie (who is no longer with us...) and Leonidas from
+ http://www.python-forum.de
* Andreas Schwarz for finding out that CaseIgnoringWordList was not case
ignoring! Such things really make you write tests.
-* closure for the first version of the Scheme scanner.
-* Stefan Walk for the first version of the JavaScript scanner.
-* Josh Goebel for another version of the JavaScript scanner and a Diff scanner.
-* Jonathan Younger for pointing out the licence confusion caused by wrong LICENSE file.
-* Jeremy Hinegardner for finding the shebang-on-empty-file bug in FileType.
-* Charles Oliver Nutter and Yehuda Katz for helping me benchmark CodeRay on JRuby.
-* Andreas Neuhaus for pointing out a markup bug in coderay/for_redcloth.
-* 0xf30fc7 for the FileType patch concerning Delphi file extensions.
-* The folks at redmine.org - thank you for using and fixing CodeRay!
* matz and all Ruby gods and gurus
* The inventors of: the computer, the internet, the true color display, HTML &
- CSS, VIM, Ruby, pizza, microwaves, guitars, scouting, programming, anime,
+ CSS, VIM, RUBY, pizza, microwaves, guitars, scouting, programming, anime,
manga, coke and green ice tea.
Where would we be without all those people?
=== Created using
* Ruby[http://ruby-lang.org/]
-* Chihiro (my Sony VAIO laptop); Henrietta (my old MacBook);
- Triella, born Rico (my new MacBook); as well as
- Seras and Hikari (my PCs)
-* RDE[http://homepage2.nifty.com/sakazuki/rde_e.html],
- VIM[http://vim.org] and TextMate[http://macromates.com]
-* Subversion[http://subversion.tigris.org/]
-* Redmine[http://redmine.org/]
-* Firefox[http://www.mozilla.org/products/firefox/],
- Firebug[http://getfirebug.com/], Safari[http://www.apple.com/safari/], and
+* Chihiro (my Sony VAIO laptop), Henrietta (my new MacBook) and
+ Seras (my Athlon 2200+ tower)
+* VIM[http://vim.org] and TextMate[http://macromates.com]
+* RDE[http://homepage2.nifty.com/sakazuki/rde_e.html]
+* Microsoft Windows (yes, I confess!) and MacOS X
+* Firefox[http://www.mozilla.org/products/firefox/] and
Thunderbird[http://www.mozilla.org/products/thunderbird/]
-* RubyGems[http://docs.rubygems.org/] and Rake[http://rake.rubyforge.org/]
-* TortoiseSVN[http://tortoisesvn.tigris.org/] using Apache via
+* Rake[http://rake.rubyforge.org/]
+* RubyGems[http://docs.rubygems.org/]
+* {Subversion/TortoiseSVN}[http://tortoisesvn.tigris.org/] using Apache via
XAMPP[http://www.apachefriends.org/en/xampp.html]
* RDoc (though I'm quite unsatisfied with it)
-* Microsoft Windows (yes, I confess!) and MacOS X
* GNUWin32, MinGW and some other tools to make the shell under windows a bit
- less useless
+ more useful
* Term::ANSIColor[http://term-ansicolor.rubyforge.org/]
-* PLEAC[http://pleac.sourceforge.net/] code examples
-=== Free
+---
* As you can see, CodeRay was created under heavy use of *free* software.
* So CodeRay is also *free*.
--- /dev/null
+#!/usr/bin/env ruby\r
+# CodeRay Executable\r
+#\r
+# Version: 0.1\r
+# Author: murphy\r
+\r
+def err msg\r
+ $stderr.puts msg\r
+end\r
+\r
+begin\r
+ require 'coderay'\r
+\r
+ if ARGV.empty?\r
+ puts <<-USAGE\r
+CodeRay #{CodeRay::VERSION} (http://rd.cYcnus.de/coderay)\r
+Usage:\r
+ coderay -<lang> [-<format>] < file > output\r
+ coderay file [-<format>]\r
+Example:\r
+ coderay -ruby -statistic < foo.rb\r
+ coderay codegen.c # generates codegen.c.html\r
+ USAGE\r
+ end\r
+\r
+ first, second = ARGV\r
+\r
+ if first\r
+ if first[/-(\w+)/] == first\r
+ lang = $1.to_sym\r
+ input = $stdin.read\r
+ tokens = :scan\r
+ elsif first == '-'\r
+ lang = $1.to_sym\r
+ input = $stdin.read\r
+ tokens = :scan\r
+ else\r
+ file = first\r
+ tokens = CodeRay.scan_file file\r
+ output_filename, output_ext = file, /#{Regexp.escape(File.extname(file))}$/\r
+ end\r
+ else\r
+ puts 'No lang/file given.'\r
+ exit 1\r
+ end\r
+\r
+ if second\r
+ if second[/-(\w+)/] == second\r
+ format = $1.to_sym\r
+ else\r
+ raise 'Invalid format (must be -xxx).'\r
+ end\r
+ else\r
+ $stderr.puts 'No format given; setting to default (HTML Page)'\r
+ format = :page\r
+ end\r
+ \r
+ # TODO: allow streaming\r
+ if tokens == :scan\r
+ output = CodeRay::Duo[lang => format].highlight input #, :stream => true\r
+ else\r
+ output = tokens.encode format\r
+ end\r
+ out = $stdout\r
+ if output_filename\r
+ output_filename += '.' + CodeRay::Encoders[format]::FILE_EXTENSION\r
+ if File.exist? output_filename\r
+ err 'File %s already exists.' % output_filename\r
+ exit\r
+ else\r
+ out = File.open output_filename, 'w'\r
+ end\r
+ end\r
+ out.print output\r
+\r
+rescue => boom\r
+ err "Error: #{boom.message}\n"\r
+ err boom.backtrace\r
+ err '-' * 50\r
+ err ARGV\r
+ exit 1\r
+end\r
--- /dev/null
+#!/usr/bin/env ruby\r
+require 'coderay'\r
+\r
+puts CodeRay::Encoders[:html]::CSS.new.stylesheet\r
# = CodeRay Library
#
+# $Id: coderay.rb 227 2007-04-24 12:26:18Z murphy $
+#
# CodeRay is a Ruby library for syntax highlighting.
#
# I try to make CodeRay easy to use and intuitive, but at the same time fully featured, complete,
#
# CodeRay.scan_stream:: Scan in stream mode.
#
-# == All-in-One Encoding
+# == All-in-One Encoding
#
# CodeRay.encode:: Highlight a string with a given input and output format.
#
module CodeRay
# Version: Major.Minor.Teeny[.Revision]
- # Major: 0 for pre-stable, 1 for stable
- # Minor: feature milestone
- # Teeny: development state, 0 for pre-release
- # Revision: Subversion Revision number (generated on rake gem:make)
- VERSION = '0.9.0'
+ # Major: 0 for pre-release
+ # Minor: odd for beta, even for stable
+ # Teeny: development state
+ # Revision: Subversion Revision number (generated on rake)
+ VERSION = '0.7.6'
require 'coderay/tokens'
- require 'coderay/token_classes'
require 'coderay/scanner'
require 'coderay/encoder'
require 'coderay/duo'
# Run a test script.
if $0 == __FILE__
$stderr.print 'Press key to print demo.'; gets
- # Just use this file as an example of Ruby code.
code = File.read(__FILE__)[/module CodeRay.*/m]
print CodeRay.scan(code, :ruby).html
end
# = Duo
#
+ # $Id: scanner.rb 123 2006-03-21 14:46:34Z murphy $
+ #
# A Duo is a convenient way to use CodeRay. You just create a Duo,
# giving it a lang (language of the input code) and a format (desired
# output format), and call Duo#highlight with the code.
+require "stringio"
+
module CodeRay
# This module holds the Encoder class and its subclasses.
# By default, it calls text_token or block_token, depending on
# whether +text+ is a String.
def token text, kind
- encoded_token =
- if text.is_a? ::String
+ out =
+ if text.is_a? ::String # Ruby 1.9: :open.is_a? String
text_token text, kind
elsif text.is_a? ::Symbol
block_token text, kind
else
raise 'Unknown token text type: %p' % text
end
- append_encoded_token_to_output encoded_token
- end
-
- def append_encoded_token_to_output encoded_token
- @out << encoded_token if encoded_token && defined?(@out) && @out
+ @out << out if @out
end
-
- # Called for each text token ([text, kind]), where text is a String.
+
def text_token text, kind
end
-
- # Called for each block (non-text) token ([action, kind]), where action is a Symbol.
+
def block_token action, kind
case action
when :open
open_token kind
when :close
close_token kind
- when :begin_line
- begin_line kind
- when :end_line
- end_line kind
else
raise 'unknown block action: %p' % action
end
end
-
- # Called for each block token at the start of the block ([:open, kind]).
- def open_token kind
- end
-
- # Called for each block token end of the block ([:close, kind]).
- def close_token kind
- end
-
- # Called for each line token block at the start of the line ([:begin_line, kind]).
- def begin_line kind
- end
-
- # Called for each line token block at the end of the line ([:end_line, kind]).
- def end_line kind
- end
# Called with merged options after encoding starts.
# The return value is the result of encoding, typically @out.
#
# The already created +tokens+ object must be used; it can be a
# TokenStream or a Tokens object.
- if RUBY_VERSION >= '1.9'
- def compile tokens, options
- for text, kind in tokens
- token text, kind
- end
- end
- else
- def compile tokens, options
- tokens.each(&self)
- end
+ def compile tokens, options
+ tokens.each(&self)
end
end
">"
end
- def begin_line kind
- "#{kind}["
- end
-
- def end_line kind
- "]"
- end
-
end
end
register_for :div
- DEFAULT_OPTIONS = HTML::DEFAULT_OPTIONS.merge \
+ DEFAULT_OPTIONS = HTML::DEFAULT_OPTIONS.merge({
:css => :style,
- :wrap => :div
+ :wrap => :div,
+ })
end
#
# == Options
#
+ # === :escape
+ # Escape html entities
+ # Default: true
+ #
# === :tab_width
# Convert \t characters to +n+ spaces (a number.)
# Default: 8
#
# Default: nil
#
- # === :title
- #
- # The title of the HTML page (works only when :wrap is set to :page.)
- #
- # Default: 'CodeRay output'
- #
# === :line_numbers
# Include line numbers in :table, :inline, :list or nil (no line numbers)
#
#
# Default: 10
#
- # === :highlight_lines
- #
- # Highlights certain line numbers now by using the :highlight_lines option.
- # Can be any Enumerable, typically just an Array or Range, of numbers.
- #
- # Bolding is deactivated when :highlight_lines is set. It only makes sense
- # in combination with :line_numbers.
- #
- # Default: nil
- #
# === :hint
# Include some information into the output using the title attribute.
# Can be :info (show token type on mouse-over), :info_long (with full path)
FILE_EXTENSION = 'html'
DEFAULT_OPTIONS = {
+ :escape => true,
:tab_width => 8,
- # :level => :xhtml, # reserved for future use
+ :level => :xhtml,
:css => :class,
:style => :cycnus,
+
:wrap => nil,
- :title => 'CodeRay output',
:line_numbers => nil,
:line_number_start => 1,
:bold_every => 10,
- :highlight_lines => nil,
:hint => false,
}
when :debug
classes.inspect
end
- title ? " title=\"#{title}\"" : ''
+ " title=\"#{title}\""
end
def setup options
@HTML_ESCAPE = HTML_ESCAPE.dup
@HTML_ESCAPE["\t"] = ' ' * options[:tab_width]
+ @escape = options[:escape]
@opened = [nil]
@css = CSS.new options[:style]
when :class
@css_style = Hash.new do |h, k|
- c = CodeRay::Tokens::ClassOfKind[k.first]
+ c = Tokens::ClassOfKind[k.first]
if c == :NO_HIGHLIGHT and not hint
h[k.dup] = false
else
@out.css = @css
@out.numerize! options[:line_numbers], options
@out.wrap! options[:wrap]
- @out.apply_title! options[:title]
super
end
- def token text, type = :plain
- case text
-
- when nil
- # raise 'Token with nil as text was given: %p' % [[text, type]]
-
- when String
- if text =~ /#{HTML_ESCAPE_PATTERN}/o
+ def token text, type
+ if text.is_a? ::String
+ if @escape && (text =~ /#{HTML_ESCAPE_PATTERN}/o)
text = text.gsub(/#{HTML_ESCAPE_PATTERN}/o) { |m| @HTML_ESCAPE[m] }
end
@opened[0] = type
- if text != "\n" && style = @css_style[@opened]
+ if style = @css_style[@opened]
@out << style << text << '</span>'
else
@out << text
end
-
-
- # token groups, eg. strings
- when :open
- @opened[0] = type
- @out << (@css_style[@opened] || '<span>')
- @opened << type
- when :close
- if @opened.empty?
- # nothing to close
- else
- if $DEBUG and (@opened.size == 1 or @opened.last != type)
- raise 'Malformed token stream: Trying to close a token (%p) \
- that is not open. Open are: %p.' % [type, @opened[1..-1]]
+ else
+ case text
+ when :open
+ @opened[0] = type
+ @out << (@css_style[@opened] || '<span>')
+ @opened << type
+ when :close
+ if @opened.empty?
+ # nothing to close
+ else
+ if $DEBUG and (@opened.size == 1 or @opened.last != type)
+ raise 'Malformed token stream: Trying to close a token (%p) \
+ that is not open. Open are: %p.' % [type, @opened[1..-1]]
+ end
+ @out << '</span>'
+ @opened.pop
end
- @out << '</span>'
- @opened.pop
- end
-
- # whole lines to be highlighted, eg. a deleted line in a diff
- when :begin_line
- @opened[0] = type
- if style = @css_style[@opened]
- @out << style.sub('<span', '<div')
+ when nil
+ raise 'Token with nil as text was given: %p' % [[text, type]]
else
- @out << '<div>'
+ raise 'unknown token kind: %p' % text
end
- @opened << type
- when :end_line
- if @opened.empty?
- # nothing to close
- else
- if $DEBUG and (@opened.size == 1 or @opened.last != type)
- raise 'Malformed token stream: Trying to close a line (%p) \
- that is not open. Open are: %p.' % [type, @opened[1..-1]]
- end
- @out << '</div>'
- @opened.pop
- end
-
- else
- raise 'unknown token kind: %p' % [text]
-
end
end
1.upto(styles.size) do |offset|
break if style = cl[styles[offset .. -1]]
end
- $stderr.puts 'Style not found: %p' % [styles] if $DEBUG and style.empty?
+ raise 'Style not found: %p' % [styles] if $DEBUG and style.empty?
return style
end
private
CSS_CLASS_PATTERN = /
- ( # $1 = selectors
- (?:
- (?: \s* \. [-\w]+ )+
- \s* ,?
- )+
- )
+ ( (?: # $1 = classes
+ \s* \. [-\w]+
+ )+ )
\s* \{ \s*
( [^\}]+ )? # $2 = style
\s* \} \s*
( . ) # $3 = error
/mx
def parse stylesheet
- stylesheet.scan CSS_CLASS_PATTERN do |selectors, style, error|
+ stylesheet.scan CSS_CLASS_PATTERN do |classes, style, error|
raise "CSS parse error: '#{error.inspect}' not recognized" if error
- for selector in selectors.split(',')
- classes = selector.scan(/[-\w]+/)
- cl = classes.pop
- @classes[cl] ||= Hash.new
- @classes[cl][classes] = style.to_s.strip.delete(' ').chomp(';')
- end
+ styles = classes.scan(/[-\w]+/)
+ cl = styles.pop
+ @classes[cl] ||= Hash.new
+ @classes[cl][styles] = style.to_s.strip
end
end
#end
bold_every = options[:bold_every]
- highlight_lines = options[:highlight_lines]
bolding =
- if bold_every == false && highlight_lines == nil
+ if bold_every == false
proc { |line| line.to_s }
- elsif highlight_lines.is_a? Enumerable
- highlight_lines = highlight_lines.to_set
- proc do |line|
- if highlight_lines.include? line
- "<strong class=\"highlighted\">#{line}</strong>" # highlighted line numbers in bold
- else
- line.to_s
- end
- end
elsif bold_every.is_a? Integer
raise ArgumentError, ":bolding can't be 0." if bold_every == 0
proc do |line|
case mode
when :inline
max_width = (start + line_count).to_s.size
- line_number = start
+ line = start
gsub!(/^/) do
- line_number_text = bolding.call line_number
- indent = ' ' * (max_width - line_number.to_s.size) # TODO: Optimize (10^x)
- res = "<span class=\"no\">#{indent}#{line_number_text}</span> "
- line_number += 1
+ line_number = bolding.call line
+ indent = ' ' * (max_width - line.to_s.size)
+ res = "<span class=\"no\">#{indent}#{line_number}</span> "
+ line += 1
res
end
# Because even monospace fonts seem to have different heights when bold,
# I make the newline bold, both in the code and the line numbers.
# FIXME Still not working perfect for Mr. Internet Exploder
+ # FIXME Firefox struggles with very long codes (> 200 lines)
line_numbers = (start ... start + line_count).to_a.map(&bolding).join("\n")
line_numbers << "\n" # also for Mr. MS Internet Exploder :-/
line_numbers.gsub!(/\n/) { "<tt>\n</tt>" }
line_numbers_table_tpl = TABLE.apply('LINE_NUMBERS', line_numbers)
- gsub!(/<\/div>\n/) { '</div>' }
gsub!(/\n/) { "<tt>\n</tt>" }
wrap_in! line_numbers_table_tpl
@wrapped_in = :div
end
close = '</span>' * opened_tags.size
- "<li>#{open}#{line}#{close}</li>\n"
+ "<li>#{open}#{line}#{close}</li>"
end
- chomp!("\n")
wrap_in! LIST
@wrapped_in = :div
Template.wrap! self, template, 'CONTENT'
self
end
-
- def apply_title! title
- self.sub!(/(<title>)(<\/title>)/) { $1 + title + $2 }
- self
- end
def wrap! element, *args
return self if not element or element == wrapped_in
wrap! :div if wrapped_in? nil
raise "Can't wrap %p in %p" % [wrapped_in, element] unless wrapped_in? :div
wrap_in! Output.page_template_for_css(@css)
- if args.first.is_a?(Hash) && title = args.first[:title]
- apply_title! title
- end
- self
when nil
return self
else
# title="double click to expand"
LIST = <<-`LIST`
-<ol class="CodeRay">
-<%CONTENT%>
-</ol>
+<ol class="CodeRay"><%CONTENT%></ol>
LIST
PAGE = <<-`PAGE`
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="de">
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8" />
- <title></title>
+ <title>CodeRay HTML Encoder Example</title>
<style type="text/css">
<%CSS%>
</style>
register_for :page
- DEFAULT_OPTIONS = HTML::DEFAULT_OPTIONS.merge \
+ DEFAULT_OPTIONS = HTML::DEFAULT_OPTIONS.merge({
:css => :class,
:wrap => :page,
:line_numbers => :table
+ })
end
register_for :span
- DEFAULT_OPTIONS = HTML::DEFAULT_OPTIONS.merge \
+ DEFAULT_OPTIONS = HTML::DEFAULT_OPTIONS.merge({
:css => :style,
- :wrap => :span
+ :wrap => :span,
+ })
end
protected
def setup options
- super
+ @out = ''
@sep = options[:separator]
end
- def text_token text, kind
- text + @sep
+ def token text, kind
+ @out << text + @sep if text.is_a? ::String
end
def finish options
- super.chomp @sep
+ @out.chomp @sep
end
end
--- /dev/null
+module CodeRay
+module Encoders
+
+ # The Tokens encoder converts the tokens to a simple
+ # readable format. It doesn't use colors and is mainly
+ # intended for console output.
+ #
+ # The tokens are converted with Tokens.write_token.
+ #
+ # The format is:
+ #
+ # <token-kind> \t <escaped token-text> \n
+ #
+ # Example:
+ #
+ # require 'coderay'
+ # puts CodeRay.scan("puts 3 + 4", :ruby).tokens
+ #
+ # prints:
+ #
+ # ident puts
+ # space
+ # integer 3
+ # space
+ # operator +
+ # space
+ # integer 4
+ #
+ class Tokens < Encoder
+
+ include Streamable
+ register_for :tokens
+
+ FILE_EXTENSION = 'tok'
+
+ protected
+ def token text, kind
+ @out << CodeRay::Tokens.write_token(text, kind)
+ end
+
+ end
+
+end
+end
end
def finish options
- @out = ''
@doc.write @out, options[:pretty], options[:transitive], true
@out
end
-#!/usr/bin/env ruby
module CodeRay
# = FileType
# That means you can get filetypes from files that don't exist.
def [] filename, read_shebang = false
name = File.basename filename
- ext = File.extname(name).sub(/^\./, '') # from last dot, delete the leading dot
- ext2 = filename.to_s[/\.(.*)/, 1] # from first dot
+ ext = File.extname name
+ ext.sub!(/^\./, '') # delete the leading dot
type =
+ TypeFromExt[ext] ||
TypeFromExt[ext.downcase] ||
- (TypeFromExt[ext2.downcase] if ext2) ||
TypeFromName[name] ||
TypeFromName[name.downcase]
type ||= shebang(filename) if read_shebang
def shebang filename
begin
File.open filename, 'r' do |f|
- if first_line = f.gets
- if type = first_line[TypeFromShebang]
- type.to_sym
- end
- end
+ first_line = f.gets
+ first_line[TypeFromShebang]
end
rescue IOError
nil
end
TypeFromExt = {
+ 'rb' => :ruby,
+ 'rbw' => :ruby,
+ 'rake' => :ruby,
+ 'mab' => :ruby,
+ 'cpp' => :c,
'c' => :c,
- 'cpp' => :cpp,
- 'css' => :css,
- 'diff' => :diff,
- 'dpr' => :delphi,
- 'groovy' => :groovy,
- 'gvy' => :groovy,
'h' => :c,
+ 'java' => :java,
+ 'js' => :javascript,
+ 'xml' => :xml,
'htm' => :html,
'html' => :html,
- 'html.erb' => :rhtml,
- 'java' => :java,
- 'js' => :java_script,
- 'json' => :json,
- 'mab' => :ruby,
- 'pas' => :delphi,
- 'patch' => :diff,
'php' => :php,
'php3' => :php,
'php4' => :php,
'php5' => :php,
- 'py' => :python,
- 'py3' => :python,
- 'pyw' => :python,
- 'rake' => :ruby,
+ 'xhtml' => :xhtml,
'raydebug' => :debug,
- 'rb' => :ruby,
- 'rbw' => :ruby,
'rhtml' => :rhtml,
- 'rxml' => :ruby,
- 'sch' => :scheme,
- 'sql' => :sql,
'ss' => :scheme,
- 'xhtml' => :xhtml,
- 'xml' => :xml,
+ 'sch' => :scheme,
'yaml' => :yaml,
'yml' => :yaml,
}
if $0 == __FILE__
$VERBOSE = true
- eval DATA.read, nil, $0, __LINE__ + 4
+ eval DATA.read, nil, $0, __LINE__+4
end
__END__
+
require 'test/unit'
-class FileTypeTests < Test::Unit::TestCase
-
- include CodeRay
-
+class TC_FileType < Test::Unit::TestCase
+
def test_fetch
assert_raise FileType::UnknownFileType do
FileType.fetch ''
def test_ruby
assert_equal :ruby, FileType['test.rb']
- assert_equal :ruby, FileType['test.java.rb']
- assert_equal :java, FileType['test.rb.java']
assert_equal :ruby, FileType['C:\\Program Files\\x\\y\\c\\test.rbw']
assert_equal :ruby, FileType['/usr/bin/something/Rakefile']
assert_equal :ruby, FileType['~/myapp/gem/Rantfile']
assert_equal :xhtml, FileType['test.xhtml']
assert_equal :xhtml, FileType['test.html.xhtml']
assert_equal :rhtml, FileType['_form.rhtml']
- assert_equal :rhtml, FileType['_form.html.erb']
end
def test_yaml
assert_not_equal :yaml, FileType['YAML']
end
- def test_pathname
- require 'pathname'
- pn = Pathname.new 'test.rb'
- assert_equal :ruby, FileType[pn]
- dir = Pathname.new '/etc/var/blubb'
- assert_equal :ruby, FileType[dir + pn]
- assert_equal :cpp, FileType[dir + 'test.cpp']
- end
-
- def test_no_shebang
+ def test_shebang
dir = './test'
if File.directory? dir
Dir.chdir dir do
end
end
end
-
- def test_shebang_empty_file
- require 'tmpdir'
- tmpfile = File.join(Dir.tmpdir, 'bla')
- File.open(tmpfile, 'w') { } # touch
- assert_equal nil, FileType[tmpfile]
- end
-
- def test_shebang
- require 'tmpdir'
- tmpfile = File.join(Dir.tmpdir, 'bla')
- File.open(tmpfile, 'w') { |f| f.puts '#!/usr/bin/env ruby' }
- assert_equal :ruby, FileType[tmpfile, true]
- end
end
#
# A simplified interface to the gzip library +zlib+ (from the Ruby Standard Library.)
#
-# Author: murphy (mail to murphy rubychan de)
+# Author: murphy (mail to murphy cYcnus de)
#
# Version: 0.2 (2005.may.28)
#
# = PluginHost
#
+# $Id: plugin.rb 220 2007-01-01 02:58:58Z murphy $
+#
# A simple subclass plugin system.
#
# Example:
#
# Generators[:fancy] #-> FancyGenerator
# # or
-# CodeRay.require_plugin 'Generators/fancy'
+# require_plugin 'Generators/fancy'
module PluginHost
# Raised if Encoders::[] fails because:
# map :navy => :dark_blue
# default :gray
# end
- def default id = nil
- if id
- id = validate_id id
- plugin_hash[nil] = id
- else
- plugin_hash[nil]
- end
+ def default id
+ id = validate_id id
+ plugin_hash[nil] = id
end
# Every plugin must register itself for one or more
def inspect
map = plugin_hash.dup
map.each do |id, plugin|
- map[id] = plugin.to_s[/(?>\w+)$/]
+ map[id] = plugin.to_s[/(?>[\w_]+)$/]
end
"#{name}[#{host_id}]#{map.inspect}"
end
id
elsif id.is_a? String
if id[/\w+/] == id
- id.downcase.to_sym
+ id.to_sym
else
raise ArgumentError, "Invalid id: '#{id}' given."
end
plugin_host.register self, *ids
end
- def title title = nil
- if title
- @title = title.to_s
- else
- @title ||= name[/([^:]+)$/, 1]
- end
- end
-
# The host for this Plugin class.
def plugin_host host = nil
if host and not host.is_a? PluginHost
#
# The above example loads the file myplugin/my_helper.rb relative to the
# file in which MyPlugin was defined.
- #
- # You can also load a helper from a different plugin:
- #
- # helper 'other_plugin/other_helper'
def helper *helpers
for helper in helpers
- if helper.is_a?(String) && helper[/\//]
- self::PLUGIN_HOST.require_helper $`, $'
- else
- self::PLUGIN_HOST.require_helper plugin_id, helper.to_s
- end
+ self::PLUGIN_HOST.require_helper plugin_id, helper.to_s
end
end
# Returns the pulgin id used by the engine.
def plugin_id
- name[/\w+$/].downcase
+ name[/[\w_]+$/].downcase
end
end
# CodeRay.require_plugin '<Host ID>/<Plugin ID>'
#
# Returns the loaded plugin.
-def self.require_plugin path
+def require_plugin path
host_id, plugin_id = path.split '/', 2
host = PluginHost.host_by_id(host_id)
raise PluginHost::HostNotFound,
h[k] = h.fetch k.downcase, default
end
else
- super(default, false)
def self.[] key # :nodoc:
super(key.downcase)
end
# = Scanners
#
+ # $Id: scanner.rb 222 2007-01-01 16:26:17Z murphy $
+ #
# This module holds the Scanner class and its subclasses.
# For example, the Ruby scanner is named CodeRay::Scanners::Ruby
# can be found in coderay/scanners/ruby.
# You can also use +map+, +any?+, +find+ and even +sort_by+,
# if you want.
class Scanner < StringScanner
-
extend Plugin
plugin_host Scanners
#
# Define @default_options for subclasses.
DEFAULT_OPTIONS = { :stream => false }
-
- KINDS_NOT_LOC = [:comment, :doctype]
class << self
end
def normify code
- code = code.to_s
- if code.respond_to? :force_encoding
- begin
- code.force_encoding 'utf-8'
- code[/\z/] # raises an ArgumentError when code contains a non-UTF-8 char
- rescue ArgumentError
- code.force_encoding 'binary'
- end
- end
- code.to_unix
+ code = code.to_s.to_unix
end
def file_extension extension = nil
else
@file_extension ||= plugin_id.to_s
end
- end
+ end
end
"but :stream is #{@options[:stream]}" if block_given?
@tokens ||= Tokens.new
end
- @tokens.scanner = self
setup
end
def line
string[0..pos].count("\n") + 1
end
-
- def column pos = self.pos
- return 0 if pos <= 0
- string = string()
- if string.respond_to?(:bytesize) && (defined?(@bin_string) || string.bytesize != string.size)
- @bin_string ||= string.dup.force_encoding(:binary)
- string = @bin_string
- end
- pos - (string.rindex(?\n, pos) || 0)
- end
protected
def reset_instance
@tokens.clear unless @options[:keep_tokens]
@cached_tokens = nil
- @bin_string = nil if defined? @bin_string
end
# Scanner error with additional status information
tokens:
%s
-current line: %d column: %d pos: %d
+current line: %d pos = %d
matched: %p state: %p
bol? = %p, eos? = %p
msg,
tokens.size,
tokens.last(10).map { |t| t.inspect }.join("\n"),
- line, column, pos,
+ line, pos,
matched, state, bol?, eos?,
- string[pos - ambit, ambit],
- string[pos, ambit],
+ string[pos-ambit,ambit],
+ string[pos,ambit],
]
end
--- /dev/null
+module CodeRay
+module Scanners
+
+ map :cpp => :c,
+ :plain => :plaintext,
+ :pascal => :delphi,
+ :irb => :ruby,
+ :xml => :html,
+ :xhtml => :nitro_xhtml,
+ :nitro => :nitro_xhtml
+
+ default :plain
+
+end
+end
class C < Scanner
- include Streamable
-
register_for :c
- file_extension 'c'
+
+ include Streamable
RESERVED_WORDS = [
- 'asm', 'break', 'case', 'continue', 'default', 'do',
- 'else', 'enum', 'for', 'goto', 'if', 'return',
- 'sizeof', 'struct', 'switch', 'typedef', 'union', 'while',
- 'restrict', # C99
+ 'asm', 'break', 'case', 'continue', 'default', 'do', 'else',
+ 'for', 'goto', 'if', 'return', 'switch', 'while',
+ 'struct', 'union', 'enum', 'typedef',
+ 'static', 'register', 'auto', 'extern',
+ 'sizeof',
+ 'volatile', 'const', # C89
+ 'inline', 'restrict', # C99
]
PREDEFINED_TYPES = [
- 'int', 'long', 'short', 'char',
+ 'int', 'long', 'short', 'char', 'void',
'signed', 'unsigned', 'float', 'double',
'bool', 'complex', # C99
]
'EOF', 'NULL',
'true', 'false', # C99
]
- DIRECTIVES = [
- 'auto', 'extern', 'register', 'static', 'void',
- 'const', 'volatile', # C89
- 'inline', # C99
- ]
IDENT_KIND = WordList.new(:ident).
add(RESERVED_WORDS, :reserved).
add(PREDEFINED_TYPES, :pre_type).
- add(DIRECTIVES, :directive).
add(PREDEFINED_CONSTANTS, :pre_constant)
- ESCAPE = / [rbfntv\n\\'"] | x[a-fA-F0-9]{1,2} | [0-7]{1,3} /x
+ ESCAPE = / [rbfnrtv\n\\'"] | x[a-fA-F0-9]{1,2} | [0-7]{1,3} /x
UNICODE_ESCAPE = / u[a-fA-F0-9]{4} | U[a-fA-F0-9]{8} /x
def scan_tokens tokens, options
match << scan_until(/ ^\# (?:elif|else|endif) .*? $ | \z /xm) unless eos?
kind = :comment
- elsif scan(/ [-+*=<>?:;,!&^|()\[\]{}~%]+ | \/=? | \.(?!\d) /x)
+ elsif scan(/ [-+*\/=<>?:;,!&^|()\[\]{}~%]+ | \.(?!\d) /x)
kind = :operator
elsif match = scan(/ [A-Za-z_][A-Za-z_0-9]* /x)
kind = IDENT_KIND[match]
if kind == :ident and check(/:(?!:)/)
- # FIXME: don't match a?b:c
+ match << scan(/:/)
kind = :label
end
- elsif scan(/\$/)
- kind = :ident
-
elsif match = scan(/L?"/)
tokens << [:open, :string]
if match[0] == ?L
elsif scan(/(?:0[0-7]+)(?![89.eEfF])/)
kind = :oct
- elsif scan(/(?:\d+)(?![.eEfF])L?L?/)
+ elsif scan(/(?:\d+)(?![.eEfF])/)
kind = :integer
elsif scan(/\d[fF]?|\d*\.\d+(?:[eE][+-]?\d+)?[fF]?|\d+[eE][+-]?\d+[fF]?/)
end
when :include_expected
- if scan(/<[^>\n]+>?|"[^"\n\\]*(?:\\.[^"\n\\]*)*"?/)
+ if scan(/[^\n]+/)
kind = :include
state = :initial
state = :initial if match.index ?\n
else
- state = :initial
- next
+ getch
+ kind = :error
end
include Streamable
register_for :debug
- file_extension 'raydebug'
- title 'CodeRay Token Dump'
protected
def scan_tokens tokens, options
class Delphi < Scanner
register_for :delphi
- file_extension 'pas'
RESERVED_WORDS = [
'and', 'array', 'as', 'at', 'asm', 'at', 'begin', 'case', 'class',
module Scanners
# HTML Scanner
+ #
+ # $Id$
class HTML < Scanner
include Streamable
register_for :html
-
- KINDS_NOT_LOC = [
- :comment, :doctype, :preprocessor,
- :tag, :attribute_name, :operator,
- :attribute_value, :delimiter, :content,
- :plain, :entity, :error
- ]
ATTR_NAME = /[\w.:-]+/
ATTR_VALUE_UNQUOTED = ATTR_NAME
if scan(/<!--.*?-->/m)
kind = :comment
elsif scan(/<!DOCTYPE.*?>/m)
- kind = :doctype
+ kind = :preprocessor
elsif scan(/<\?xml.*?\?>/m)
kind = :preprocessor
elsif scan(/<\?.*?\?>|<%.*?%>/m)
kind = :comment
- elsif scan(/<\/[-\w.:]*>/m)
+ elsif scan(/<\/[-\w_.:]*>/m)
kind = :tag
- elsif match = scan(/<[-\w.:]+>?/m)
+ elsif match = scan(/<[-\w_.:]+>?/m)
kind = :tag
state = :attribute unless match[-1] == ?>
elsif scan(/[^<>&]+/)
--- /dev/null
+module CodeRay
+ module Scanners
+ class Java < Scanner
+
+ register_for :java
+
+ RESERVED_WORDS = %w(abstract assert break case catch class
+ const continue default do else enum extends final finally for
+ goto if implements import instanceof interface native new
+ package private protected public return static strictfp super switch
+ synchronized this throw throws transient try void volatile while)
+
+ PREDEFINED_TYPES = %w(boolean byte char double float int long short)
+
+ PREDEFINED_CONSTANTS = %w(true false null)
+
+ IDENT_KIND = WordList.new(:ident).
+ add(RESERVED_WORDS, :reserved).
+ add(PREDEFINED_TYPES, :pre_type).
+ add(PREDEFINED_CONSTANTS, :pre_constant)
+
+ ESCAPE = / [rbfnrtv\n\\'"] | x[a-fA-F0-9]{1,2} | [0-7]{1,3} /x
+ UNICODE_ESCAPE = / u[a-fA-F0-9]{4} | U[a-fA-F0-9]{8} /x
+
+ def scan_tokens tokens, options
+ state = :initial
+
+ until eos?
+ kind = nil
+ match = nil
+
+ case state
+ when :initial
+
+ if scan(/ \s+ | \\\n /x)
+ kind = :space
+
+ elsif scan(%r! // [^\n\\]* (?: \\. [^\n\\]* )* | /\* (?: .*? \*/ | .* ) !mx)
+ kind = :comment
+
+ elsif match = scan(/ \# \s* if \s* 0 /x)
+ match << scan_until(/ ^\# (?:elif|else|endif) .*? $ | \z /xm) unless eos?
+ kind = :comment
+
+ elsif scan(/ [-+*\/=<>?:;,!&^|()\[\]{}~%]+ | \.(?!\d) /x)
+ kind = :operator
+
+ elsif match = scan(/ [A-Za-z_][A-Za-z_0-9]* /x)
+ kind = IDENT_KIND[match]
+ if kind == :ident and check(/:(?!:)/)
+ match << scan(/:/)
+ kind = :label
+ end
+
+ elsif match = scan(/L?"/)
+ tokens << [:open, :string]
+ if match[0] == ?L
+ tokens << ['L', :modifier]
+ match = '"'
+ end
+ state = :string
+ kind = :delimiter
+
+ elsif scan(%r! \@ .* !x)
+ kind = :preprocessor
+
+ elsif scan(/ L?' (?: [^\'\n\\] | \\ #{ESCAPE} )? '? /ox)
+ kind = :char
+
+ elsif scan(/0[xX][0-9A-Fa-f]+/)
+ kind = :hex
+
+ elsif scan(/(?:0[0-7]+)(?![89.eEfF])/)
+ kind = :oct
+
+ elsif scan(/(?:\d+)(?![.eEfF])/)
+ kind = :integer
+
+ elsif scan(/\d[fF]?|\d*\.\d+(?:[eE][+-]?\d+)?[fF]?|\d+[eE][+-]?\d+[fF]?/)
+ kind = :float
+
+ else
+ getch
+ kind = :error
+
+ end
+
+ when :string
+ if scan(/[^\\\n"]+/)
+ kind = :content
+ elsif scan(/"/)
+ tokens << ['"', :delimiter]
+ tokens << [:close, :string]
+ state = :initial
+ next
+ elsif scan(/ \\ (?: #{ESCAPE} | #{UNICODE_ESCAPE} ) /mox)
+ kind = :char
+ elsif scan(/ \\ | $ /x)
+ tokens << [:close, :string]
+ kind = :error
+ state = :initial
+ else
+ raise_inspect "else case \" reached; %p not handled." % peek(1), tokens
+ end
+
+ else
+ raise_inspect 'Unknown state', tokens
+
+ end
+
+ match ||= matched
+ if $DEBUG and not kind
+ raise_inspect 'Error token %p in line %d' %
+ [[match, kind], line], tokens
+ end
+ raise_inspect 'Empty token', tokens unless match
+
+ tokens << [match, kind]
+
+ end
+
+ if state == :string
+ tokens << [:close, :string]
+ end
+
+ tokens
+ end
+ end
+ end
+end
--- /dev/null
+# http://pastie.textmate.org/50774/\r
+module CodeRay module Scanners\r
+ \r
+ class JavaScript < Scanner\r
+\r
+ register_for :javascript\r
+ \r
+ RESERVED_WORDS = [\r
+ 'asm', 'break', 'case', 'continue', 'default', 'do', 'else',\r
+ 'for', 'goto', 'if', 'return', 'switch', 'while',\r
+# 'struct', 'union', 'enum', 'typedef',\r
+# 'static', 'register', 'auto', 'extern',\r
+# 'sizeof',\r
+ 'typeof',\r
+# 'volatile', 'const', # C89\r
+# 'inline', 'restrict', # C99 \r
+ 'var', 'function','try','new','in',\r
+ 'instanceof','throw','catch'\r
+ ]\r
+\r
+ PREDEFINED_CONSTANTS = [\r
+ 'void', 'null', 'this',\r
+ 'true', 'false','undefined',\r
+ ]\r
+\r
+ IDENT_KIND = WordList.new(:ident).\r
+ add(RESERVED_WORDS, :reserved).\r
+ add(PREDEFINED_CONSTANTS, :pre_constant)\r
+\r
+ ESCAPE = / [rbfnrtv\n\\\/'"] | x[a-fA-F0-9]{1,2} | [0-7]{1,3} /x\r
+ UNICODE_ESCAPE = / u[a-fA-F0-9]{4} | U[a-fA-F0-9]{8} /x\r
+\r
+ def scan_tokens tokens, options\r
+\r
+ state = :initial\r
+ string_type = nil\r
+ regexp_allowed = true\r
+\r
+ until eos?\r
+\r
+ kind = :error\r
+ match = nil\r
+\r
+ if state == :initial\r
+ \r
+ if scan(/ \s+ | \\\n /x)\r
+ kind = :space\r
+ \r
+ elsif scan(%r! // [^\n\\]* (?: \\. [^\n\\]* )* | /\* (?: .*? \*/ | .* ) !mx)\r
+ kind = :comment\r
+ regexp_allowed = false\r
+\r
+ elsif match = scan(/ \# \s* if \s* 0 /x)\r
+ match << scan_until(/ ^\# (?:elif|else|endif) .*? $ | \z /xm) unless eos?\r
+ kind = :comment\r
+ regexp_allowed = false\r
+\r
+ elsif regexp_allowed and scan(/\//)\r
+ tokens << [:open, :regexp]\r
+ state = :regex\r
+ kind = :delimiter\r
+ \r
+ elsif scan(/ [-+*\/=<>?:;,!&^|()\[\]{}~%] | \.(?!\d) /x)\r
+ kind = :operator\r
+ regexp_allowed=true\r
+ \r
+ elsif match = scan(/ [$A-Za-z_][A-Za-z_0-9]* /x)\r
+ kind = IDENT_KIND[match]\r
+# if kind == :ident and check(/:(?!:)/)\r
+# match << scan(/:/)\r
+# kind = :label\r
+# end\r
+ regexp_allowed=false\r
+ \r
+ elsif match = scan(/["']/)\r
+ tokens << [:open, :string]\r
+ string_type = matched\r
+ state = :string\r
+ kind = :delimiter\r
+ \r
+# elsif scan(/#\s*(\w*)/)\r
+# kind = :preprocessor # FIXME multiline preprocs\r
+# state = :include_expected if self[1] == 'include'\r
+# \r
+# elsif scan(/ L?' (?: [^\'\n\\] | \\ #{ESCAPE} )? '? /ox)\r
+# kind = :char\r
+ \r
+ elsif scan(/0[xX][0-9A-Fa-f]+/)\r
+ kind = :hex\r
+ regexp_allowed=false\r
+ \r
+ elsif scan(/(?:0[0-7]+)(?![89.eEfF])/)\r
+ kind = :oct\r
+ regexp_allowed=false\r
+ \r
+ elsif scan(/(?:\d+)(?![.eEfF])/)\r
+ kind = :integer\r
+ regexp_allowed=false\r
+ \r
+ elsif scan(/\d[fF]?|\d*\.\d+(?:[eE][+-]?\d+)?[fF]?|\d+[eE][+-]?\d+[fF]?/)\r
+ kind = :float\r
+ regexp_allowed=false\r
+\r
+ else\r
+ getch\r
+ end\r
+ \r
+ elsif state == :regex\r
+ if scan(/[^\\\/]+/)\r
+ kind = :content\r
+ elsif scan(/\\\/|\\\\/)\r
+ kind = :content\r
+ elsif scan(/\//)\r
+ tokens << [matched, :delimiter]\r
+ tokens << [:close, :regexp]\r
+ state = :initial\r
+ next\r
+ else\r
+ getch\r
+ kind = :content\r
+ end\r
+ \r
+ elsif state == :string\r
+ if scan(/[^\\"']+/)\r
+ kind = :content\r
+ elsif scan(/["']/)\r
+ if string_type==matched\r
+ tokens << [matched, :delimiter]\r
+ tokens << [:close, :string]\r
+ state = :initial\r
+ string_type=nil\r
+ next\r
+ else\r
+ kind = :content\r
+ end\r
+ elsif scan(/ \\ (?: #{ESCAPE} | #{UNICODE_ESCAPE} ) /mox)\r
+ kind = :char\r
+ elsif scan(/ \\ | $ /x)\r
+ kind = :error\r
+ state = :initial\r
+ else\r
+ raise "else case \" reached; %p not handled." % peek(1), tokens\r
+ end\r
+ \r
+# elsif state == :include_expected\r
+# if scan(/<[^>\n]+>?|"[^"\n\\]*(?:\\.[^"\n\\]*)*"?/)\r
+# kind = :include\r
+# state = :initial\r
+# \r
+# elsif match = scan(/\s+/)\r
+# kind = :space\r
+# state = :initial if match.index ?\n\r
+# \r
+# else\r
+# getch\r
+# \r
+# end\r
+# \r
+ else\r
+ raise 'else-case reached', tokens\r
+ \r
+ end\r
+ \r
+ match ||= matched\r
+# raise [match, kind], tokens if kind == :error\r
+ \r
+ tokens << [match, kind]\r
+ \r
+ end\r
+ tokens\r
+ \r
+ end\r
+\r
+ end\r
+\r
+end end
\ No newline at end of file
load :ruby
# Nitro XHTML Scanner
+ #
+ # $Id$
class NitroXHTML < Scanner
include Streamable
register_for :nitro_xhtml
- file_extension :xhtml
- title 'Nitro XHTML'
- KINDS_NOT_LOC = HTML::KINDS_NOT_LOC
-
NITRO_RUBY_BLOCK = /
<\?r
(?>
--- /dev/null
+module CodeRay module Scanners
+
+ class PHP < Scanner
+
+ register_for :php
+
+ RESERVED_WORDS = [
+ 'and', 'or', 'xor', '__FILE__', 'exception', '__LINE__', 'array', 'as', 'break', 'case',
+ 'class', 'const', 'continue', 'declare', 'default',
+ 'die', 'do', 'echo', 'else', 'elseif',
+ 'empty', 'enddeclare', 'endfor', 'endforeach', 'endif',
+ 'endswitch', 'endwhile', 'eval', 'exit', 'extends',
+ 'for', 'foreach', 'function', 'global', 'if',
+ 'include', 'include_once', 'isset', 'list', 'new',
+ 'print', 'require', 'require_once', 'return', 'static',
+ 'switch', 'unset', 'use', 'var', 'while',
+ '__FUNCTION__', '__CLASS__', '__METHOD__', 'final', 'php_user_filter',
+ 'interface', 'implements', 'extends', 'public', 'private',
+ 'protected', 'abstract', 'clone', 'try', 'catch',
+ 'throw', 'cfunction', 'old_function'
+ ]
+
+ PREDEFINED_CONSTANTS = [
+ 'null', '$this', 'true', 'false'
+ ]
+
+ IDENT_KIND = WordList.new(:ident).
+ add(RESERVED_WORDS, :reserved).
+ add(PREDEFINED_CONSTANTS, :pre_constant)
+
+ ESCAPE = / [\$\wrbfnrtv\n\\\/'"] | x[a-fA-F0-9]{1,2} | [0-7]{1,3} /x
+ UNICODE_ESCAPE = / u[a-fA-F0-9]{4} | U[a-fA-F0-9]{8} /x
+
+ def scan_tokens tokens, options
+
+ state = :waiting_php
+ string_type = nil
+ regexp_allowed = true
+
+ until eos?
+
+ kind = :error
+ match = nil
+
+ if state == :initial
+
+ if scan(/ \s+ | \\\n /x)
+ kind = :space
+
+ elsif scan(/\?>/)
+ kind = :char
+ state = :waiting_php
+
+ elsif scan(%r{ (//|\#) [^\n\\]* (?: \\. [^\n\\]* )* | /\* (?: .*? \*/ | .* ) }mx)
+ kind = :comment
+ regexp_allowed = false
+
+ elsif match = scan(/ \# \s* if \s* 0 /x)
+ match << scan_until(/ ^\# (?:elif|else|endif) .*? $ | \z /xm) unless eos?
+ kind = :comment
+ regexp_allowed = false
+
+ elsif regexp_allowed and scan(/\//)
+ tokens << [:open, :regexp]
+ state = :regex
+ kind = :delimiter
+
+ elsif scan(/ [-+*\/=<>?:;,!&^|()\[\]{}~%] | \.(?!\d) /x)
+ kind = :operator
+ regexp_allowed=true
+
+ elsif match = scan(/ [$@A-Za-z_][A-Za-z_0-9]* /x)
+ kind = IDENT_KIND[match]
+ regexp_allowed=false
+
+ elsif match = scan(/["']/)
+ tokens << [:open, :string]
+ string_type = matched
+ state = :string
+ kind = :delimiter
+
+ elsif scan(/0[xX][0-9A-Fa-f]+/)
+ kind = :hex
+ regexp_allowed=false
+
+ elsif scan(/(?:0[0-7]+)(?![89.eEfF])/)
+ kind = :oct
+ regexp_allowed=false
+
+ elsif scan(/(?:\d+)(?![.eEfF])/)
+ kind = :integer
+ regexp_allowed=false
+
+ elsif scan(/\d[fF]?|\d*\.\d+(?:[eE][+-]?\d+)?[fF]?|\d+[eE][+-]?\d+[fF]?/)
+ kind = :float
+ regexp_allowed=false
+
+ else
+ getch
+ end
+
+ elsif state == :regex
+ if scan(/[^\\\/]+/)
+ kind = :content
+ elsif scan(/\\\/|\\/)
+ kind = :content
+ elsif scan(/\//)
+ tokens << [matched, :delimiter]
+ tokens << [:close, :regexp]
+ state = :initial
+ next
+ else
+ getch
+ kind = :content
+ end
+
+ elsif state == :string
+ if scan(/[^\\"']+/)
+ kind = :content
+ elsif scan(/["']/)
+ if string_type==matched
+ tokens << [matched, :delimiter]
+ tokens << [:close, :string]
+ state = :initial
+ string_type=nil
+ next
+ else
+ kind = :content
+ end
+ elsif scan(/ \\ (?: \S ) /mox)
+ kind = :char
+ elsif scan(/ \\ | $ /x)
+ kind = :error
+ state = :initial
+ else
+ raise "else case \" reached; %p not handled." % peek(1), tokens
+ end
+
+ elsif state == :waiting_php
+ if scan(/<\?php/m)
+ kind = :char
+ state = :initial
+ elsif scan(/[^<]+/)
+ kind = :comment
+ else
+ kind = :comment
+ getch
+ end
+ else
+ raise 'else-case reached', tokens
+
+ end
+
+ match ||= matched
+
+ tokens << [match, kind]
+
+ end
+ tokens
+
+ end
+
+ end
+
+end end
\ No newline at end of file
class Plaintext < Scanner
register_for :plaintext, :plain
- title 'Plain text'
include Streamable
-
- KINDS_NOT_LOC = [:plain]
-
+
def scan_tokens tokens, options
text = (scan_until(/\z/) || '')
tokens << [text, :plain]
load :ruby
# RHTML Scanner
+ #
+ # $Id$
class RHTML < Scanner
include Streamable
register_for :rhtml
- title 'HTML ERB Template'
-
- KINDS_NOT_LOC = HTML::KINDS_NOT_LOC
ERB_RUBY_BLOCK = /
<%(?!%)[=-]?
file_extension 'rb'
helper :patterns
-
- if not defined? EncodingError
- EncodingError = Class.new Exception
- end
private
def scan_tokens tokens, options
state = :initial
depth = nil
inline_block_stack = []
- unicode = string.respond_to?(:encoding) && string.encoding.name == 'UTF-8'
-
+
patterns = Patterns # avoid constant lookup
-
+
until eos?
match = nil
kind = nil
# {{{
if match = scan(/[ \t\f]+/)
kind = :space
- match << scan(/\s*/) unless eos? || heredocs
- value_expected = true if match.index(?\n)
+ match << scan(/\s*/) unless eos? or heredocs
tokens << [match, kind]
next
elsif match = scan(/\\?\n/)
kind = :space
if match == "\n"
- value_expected = true
+ value_expected = true # FIXME not quite true
state = :initial if state == :undef_comma_expected
end
if heredocs
tokens << [match, kind]
next
- elsif bol? && match = scan(/\#!.*/)
- tokens << [match, :doctype]
- next
-
elsif match = scan(/\#.*/) or
( bol? and match = scan(/#{patterns::RUBYDOC_OR_DATA}/o) )
kind = :comment
+ value_expected = true
tokens << [match, kind]
next
elsif state == :initial
# IDENTS #
- if match = scan(unicode ? /#{patterns::METHOD_NAME}/uo :
- /#{patterns::METHOD_NAME}/o)
+ if match = scan(/#{patterns::METHOD_NAME}/o)
if last_token_dot
kind = if match[/^[A-Z]/] and not match?(/\(/) then :constant else :ident end
else
kind = :constant
elsif kind == :reserved
state = patterns::DEF_NEW_STATE[match]
- value_expected = :set if patterns::VALUE_EXPECTING_KEYWORDS[match]
end
end
- value_expected = :set if check(/#{patterns::VALUE_FOLLOWS}/o)
+ ## experimental!
+ value_expected = :set if
+ patterns::REGEXP_ALLOWED[match] or check(/#{patterns::VALUE_FOLLOWS}/o)
- elsif last_token_dot and match = scan(/#{patterns::METHOD_NAME_OPERATOR}|\(/o)
+ elsif last_token_dot and match = scan(/#{patterns::METHOD_NAME_OPERATOR}/o)
kind = :ident
value_expected = :set if check(/#{patterns::VALUE_FOLLOWS}/o)
depth -= 1
if depth == 0 # closing brace of inline block reached
state, depth, heredocs = inline_block_stack.pop
- heredocs = nil if heredocs && heredocs.empty?
tokens << [match, :inline_delimiter]
kind = :inline
match = :close
interpreted = true
state = patterns::StringState.new :regexp, interpreted, match
- # elsif match = scan(/[-+]?#{patterns::NUMERIC}/o)
- elsif match = value_expected ? scan(/[-+]?#{patterns::NUMERIC}/o) : scan(/#{patterns::NUMERIC}/o)
- kind = self[1] ? :float : :integer
+ elsif match = scan(/#{patterns::NUMERIC}/o)
+ kind = if self[1] then :float else :integer end
elsif match = scan(/#{patterns::SYMBOL}/o)
case delim = match[1]
else
kind = :error
- match = (scan(/./mu) rescue nil) || getch
- if !unicode && match.size > 1
- # warn 'Switchig to unicode mode: %p' % ['ä'[/#{patterns::METHOD_NAME}/uo]]
- unicode = true
- unscan
- next
- end
+ match = getch
end
elsif state == :def_expected
state = :initial
- if scan(/self\./)
- tokens << ['self', :pre_constant]
- tokens << ['.', :operator]
- end
- if match = scan(unicode ? /(?>#{patterns::METHOD_NAME_EX})(?!\.|::)/uo :
- /(?>#{patterns::METHOD_NAME_EX})(?!\.|::)/o)
+ if match = scan(/(?>#{patterns::METHOD_NAME_EX})(?!\.|::)/o)
kind = :method
else
next
end
- elsif state == :module_expected
- if match = scan(/<</)
- kind = :operator
- else
- state = :initial
- if match = scan(/ (?:#{patterns::IDENT}::)* #{patterns::IDENT} /ox)
- kind = :class
- else
- next
- end
- end
-
elsif state == :undef_expected
state = :undef_comma_expected
if match = scan(/#{patterns::METHOD_NAME_EX}/o)
next
end
- elsif state == :alias_expected
- begin
- match = scan(unicode ? /(#{patterns::METHOD_NAME_OR_SYMBOL})([ \t]+)(#{patterns::METHOD_NAME_OR_SYMBOL})/uo :
- /(#{patterns::METHOD_NAME_OR_SYMBOL})([ \t]+)(#{patterns::METHOD_NAME_OR_SYMBOL})/o)
- rescue EncodingError
- raise if $DEBUG
- end
-
- if match
- tokens << [self[1], (self[1][0] == ?: ? :symbol : :method)]
- tokens << [self[2], :space]
- tokens << [self[3], (self[3][0] == ?: ? :symbol : :method)]
- end
- state = :initial
- next
-
elsif state == :undef_comma_expected
if match = scan(/,/)
kind = :operator
next
end
+ elsif state == :module_expected
+ if match = scan(/<</)
+ kind = :operator
+ else
+ state = :initial
+ if match = scan(/ (?:#{patterns::IDENT}::)* #{patterns::IDENT} /ox)
+ kind = :class
+ else
+ next
+ end
+ end
+
end
# }}}
-
- unless kind == :error
- value_expected = value_expected == :set
- last_token_dot = last_token_dot == :set
- end
-
+
+ value_expected = value_expected == :set
+ last_token_dot = last_token_dot == :set
+
if $DEBUG and not kind
raise_inspect 'Error token %p in line %d' %
[[match, kind], line], tokens, state
-# encoding: utf-8
module CodeRay
module Scanners
DEF_KEYWORDS = %w[ def ]
UNDEF_KEYWORDS = %w[ undef ]
- ALIAS_KEYWORDS = %w[ alias ]
MODULE_KEYWORDS = %w[class module]
DEF_NEW_STATE = WordList.new(:initial).
add(DEF_KEYWORDS, :def_expected).
add(UNDEF_KEYWORDS, :undef_expected).
- add(ALIAS_KEYWORDS, :alias_expected).
add(MODULE_KEYWORDS, :module_expected)
+ IDENTS_ALLOWING_REGEXP = %w[
+ and or not while until unless if then elsif when sub sub! gsub gsub!
+ scan slice slice! split
+ ]
+ REGEXP_ALLOWED = WordList.new(false).
+ add(IDENTS_ALLOWING_REGEXP, :set)
+
PREDEFINED_CONSTANTS = %w[
nil true false self
DATA ARGV ARGF __FILE__ __LINE__
add(RESERVED_WORDS, :reserved).
add(PREDEFINED_CONSTANTS, :pre_constant)
- IDENT = 'ä'[/[[:alpha:]]/] == 'ä' ? /[[:alpha:]_][[:alnum:]_]*/ : /[^\W\d]\w*/
+ IDENT = /[a-z_][\w_]*/i
METHOD_NAME = / #{IDENT} [?!]? /ox
METHOD_NAME_OPERATOR = /
\*\*? # multiplication and power
- | [-+~]@? # plus, minus, tilde with and without at sign
- | [\/%&|^`] # division, modulo or format strings, and, or, xor, system
+ | [-+]@? # plus, minus
+ | [\/%&|^`~] # division, modulo or format strings, &and, |or, ^xor, `system`, tilde
| \[\]=? # array getter and setter
| << | >> # append or shift left, shift right
| <=?>? | >=? # comparison, rocket operator
- | ===? | =~ # simple equality, case equality, match
- | ![~=@]? # negation with and without at sign, not-equal and not-match
+ | ===? # simple equality and case equality
/ox
METHOD_NAME_EX = / #{IDENT} (?:[?!]|=(?!>))? | #{METHOD_NAME_OPERATOR} /ox
INSTANCE_VARIABLE = / @ #{IDENT} /ox
CLASS_VARIABLE = / @@ #{IDENT} /ox
OBJECT_VARIABLE = / @@? #{IDENT} /ox
GLOBAL_VARIABLE = / \$ (?: #{IDENT} | [1-9]\d* | 0\w* | [~&+`'=\/,;_.<>!@$?*":\\] | -[a-zA-Z_0-9] ) /ox
- PREFIX_VARIABLE = / #{GLOBAL_VARIABLE} | #{OBJECT_VARIABLE} /ox
+ PREFIX_VARIABLE = / #{GLOBAL_VARIABLE} |#{OBJECT_VARIABLE} /ox
VARIABLE = / @?@? #{IDENT} | #{GLOBAL_VARIABLE} /ox
QUOTE_TO_TYPE = {
QUOTE_TO_TYPE.default = :string
REGEXP_MODIFIERS = /[mixounse]*/
- REGEXP_SYMBOLS = /[|?*+(){}\[\].^$]/
+ REGEXP_SYMBOLS = /[|?*+?(){}\[\].^$]/
DECIMAL = /\d+(?:_\d+)*/
OCTAL = /0_?[0-7]+(?:_[0-7]+)*/
EXPONENT = / [eE] [+-]? #{DECIMAL} /ox
FLOAT_SUFFIX = / #{EXPONENT} | \. #{DECIMAL} #{EXPONENT}? /ox
FLOAT_OR_INT = / #{DECIMAL} (?: #{FLOAT_SUFFIX} () )? /ox
- NUMERIC = / (?: (?=0) (?: #{OCTAL} | #{HEXADECIMAL} | #{BINARY} ) | #{FLOAT_OR_INT} ) /ox
+ NUMERIC = / [-+]? (?: (?=0) (?: #{OCTAL} | #{HEXADECIMAL} | #{BINARY} ) | #{FLOAT_OR_INT} ) /ox
SYMBOL = /
:
| ['"]
)
/ox
- METHOD_NAME_OR_SYMBOL = / #{METHOD_NAME_EX} | #{SYMBOL} /ox
- SIMPLE_ESCAPE = /
+ # TODO investigste \M, \c and \C escape sequences
+ # (?: M-\\C-|C-\\M-|M-\\c|c\\M-|c|C-|M-)? (?: \\ (?: [0-7]{3} | x[0-9A-Fa-f]{2} | . ) )
+ # assert_equal(225, ?\M-a)
+ # assert_equal(129, ?\M-\C-a)
+ ESCAPE = /
[abefnrstv]
+ | M-\\C-|C-\\M-|M-\\c|c\\M-|c|C-|M-
| [0-7]{1,3}
| x[0-9A-Fa-f]{1,2}
- | .?
+ | .
/mx
-
- CONTROL_META_ESCAPE = /
- (?: M-|C-|c )
- (?: \\ (?: M-|C-|c ) )*
- (?: [^\\] | \\ #{SIMPLE_ESCAPE} )?
- /mox
-
- ESCAPE = /
- #{CONTROL_META_ESCAPE} | #{SIMPLE_ESCAPE}
- /mox
-
+
CHARACTER = /
\?
(?:
[^\s\\]
| \\ #{ESCAPE}
)
- /mox
+ /mx
# NOTE: This is not completely correct, but
# nobody needs heredoc delimiters ending with \n.
/mx
# Checks for a valid value to follow. This enables
- # value_expected in method calls without parentheses.
+ # fancy_allowed in method calls.
VALUE_FOLLOWS = /
- (?>[ \t\f\v]+)
+ \s+
(?:
[%\/][^\s=]
- | <<-?\S
- | [-+] \d
- | #{CHARACTER}
+ |
+ <<-?\S
+ |
+ #{CHARACTER}
)
/x
- VALUE_EXPECTING_KEYWORDS = WordList.new.add(%w[
- and end in or unless begin
- defined? ensure redo super until
- break do next rescue then
- when case else for retry
- while elsif if not return
- yield
- ])
RUBYDOC_OR_DATA = / #{RUBYDOC} | #{DATA} /xo
RDOC_DATA_START = / ^=begin (?!\S) | ^__END__$ /x
+ # FIXME: \s and = are only a workaround, they are still allowed
+ # as delimiters.
+ FANCY_START_SAVE = / % ( [qQwWxsr] | (?![a-zA-Z0-9\s=]) ) ([^a-zA-Z0-9]) /mx
FANCY_START_CORRECT = / % ( [qQwWxsr] | (?![a-zA-Z0-9]) ) ([^a-zA-Z0-9]) /mx
FancyStringType = {
{ }
] ]
- CLOSING_PAREN.each { |k,v| k.freeze; v.freeze } # debug, if I try to change it with <<
+ CLOSING_PAREN.values.each { |o| o.freeze } # debug, if I try to change it with <<
OPENING_PAREN = CLOSING_PAREN.invert
- STRING_PATTERN = Hash.new do |h, k|
+ STRING_PATTERN = Hash.new { |h, k|
delim, interpreted = *k
- delim_pattern = Regexp.escape(delim)
+ delim_pattern = Regexp.escape(delim.dup)
if closing_paren = CLOSING_PAREN[delim]
- delim_pattern = delim_pattern[0..-1] if defined? JRUBY_VERSION # JRuby fix
delim_pattern << Regexp.escape(closing_paren)
end
- delim_pattern << '\\\\' unless delim == '\\'
-
+
+
special_escapes =
case interpreted
when :regexp_symbols
when :words
'| \s'
end
-
+
h[k] =
if interpreted and not delim == '#'
- / (?= [#{delim_pattern}] | \# [{$@] #{special_escapes} ) /mx
+ / (?= [#{delim_pattern}\\] | \# [{$@] #{special_escapes} ) /mx
else
- / (?= [#{delim_pattern}] #{special_escapes} ) /mx
+ / (?= [#{delim_pattern}\\] #{special_escapes} ) /mx
end
- end
+ }
- HEREDOC_PATTERN = Hash.new do |h, k|
+ HEREDOC_PATTERN = Hash.new { |h, k|
delim, interpreted, indented = *k
delim_pattern = Regexp.escape(delim.dup)
delim_pattern = / \n #{ '(?>[\ \t]*)' if indented } #{ Regexp.new delim_pattern } $ /x
else
/ (?= #{delim_pattern}() | \\ ) /mx
end
- end
+ }
def initialize kind, interpreted, delim, heredoc = false
if heredoc
pattern = HEREDOC_PATTERN[ [delim, interpreted, heredoc == :indented] ]
- delim = nil
+ delim = nil
else
pattern = STRING_PATTERN[ [delim, interpreted] ]
if paren = CLOSING_PAREN[delim]
class Scheme < Scanner
register_for :scheme
- file_extension 'scm'
+ file_extension :scm
CORE_FORMS = %w[
lambda let let* letrec syntax-case define-syntax let-syntax
# XML Scanner
#
+ # $Id$
+ #
# Currently this is the same scanner as Scanners::HTML.
class XML < HTML
register_for :xml
- file_extension 'xml'
-
+
end
end
code_background = '#f8f8f8'
numbers_background = '#def'
border_color = 'silver'
- normal_color = '#000'
+ normal_color = '#100'
CSS_MAIN_STYLES = <<-MAIN
.CodeRay {
text-align: right;
}
.CodeRay .line_numbers tt { font-weight: bold }
-.CodeRay .line_numbers .highlighted { color: red }
.CodeRay .no { padding: 0px 4px }
.CodeRay .code { width: 100% }
.af { color:#00C }
.an { color:#007 }
-.at { color:#f08 }
.av { color:#700 }
.aw { color:#C00 }
.bi { color:#509; font-weight:bold }
-.c { color:#888; }
+.c { color:#666; }
.ch { color:#04D }
.ch .k { color:#04D }
.ch .dl { color:#039 }
.cl { color:#B06; font-weight:bold }
-.cm { color:#A08; font-weight:bold }
.co { color:#036; font-weight:bold }
.cr { color:#0A0 }
.cv { color:#369 }
-.de { color:#B0B; }
.df { color:#099; font-weight:bold }
.di { color:#088; font-weight:bold }
.dl { color:black }
.do { color:#970 }
-.dt { color:#34b }
.ds { color:#D42; font-weight:bold }
.e { color:#666; font-weight:bold }
.en { color:#800; font-weight:bold }
.er { color:#F00; background-color:#FAA }
-.ex { color:#C00; font-weight:bold }
+.ex { color:#F00; font-weight:bold }
.fl { color:#60E; font-weight:bold }
.fu { color:#06B; font-weight:bold }
.gv { color:#d70; font-weight:bold }
.i { color:#00D; font-weight:bold }
.ic { color:#B44; font-weight:bold }
-.il { background: #ddd; color: black }
-.il .il { background: #ccc }
-.il .il .il { background: #bbb }
-.il .idl { background: #ddd; font-weight: bold; color: #666 }
-.idl { background-color: #bbb; font-weight: bold; color: #666; }
+.il { background: #eee }
+.il .il { background: #ddd }
+.il .il .il { background: #ccc }
+.il .idl { font-weight: bold; color: #888 }
-.im { color:#f00; }
.in { color:#B2B; font-weight:bold }
.iv { color:#33B }
.la { color:#970; font-weight:bold }
.op { }
.pc { color:#038; font-weight:bold }
.pd { color:#369; font-weight:bold }
-.pp { color:#579; }
-.ps { color:#00C; font-weight:bold; }
-.pt { color:#074; font-weight:bold }
-.r, .kw { color:#080; font-weight:bold }
-
-.ke { color: #808; }
-.ke .dl { color: #606; }
-.ke .ch { color: #80f; }
-.vl { color: #088; }
+.pp { color:#579 }
+.pt { color:#339; font-weight:bold }
+.r { color:#080; font-weight:bold }
.rx { background-color:#fff0ff }
.rx .k { color:#808 }
.rx .mod { color:#C2C }
.rx .fu { color:#404; font-weight: bold }
-.s { background-color:#fff0f0; color: #D20; }
-.s .s { background-color:#ffe0e0 }
-.s .s .s { background-color:#ffd0d0 }
-.s .k { }
-.s .ch { color: #b0b; }
-.s .dl { color: #710; }
+.s { background-color:#fff0f0 }
+.s .s { background-color:#ffe0e0 }
+.s .s .s { background-color:#ffd0d0 }
+.s .k { color:#D20 }
+.s .dl { color:#710 }
-.sh { background-color:#f0fff0; color:#2B2 }
-.sh .k { }
+.sh { background-color:#f0fff0 }
+.sh .k { color:#2B2 }
.sh .dl { color:#161 }
.sy { color:#A60 }
.ty { color:#339; font-weight:bold }
.v { color:#036 }
.xt { color:#444 }
-
-.ins { background: #afa; }
-.del { background: #faa; }
-.chg { color: #aaf; background: #007; }
-.head { color: #f8f; background: #505 }
-
-.ins .ins { color: #080; font-weight:bold }
-.del .del { color: #800; font-weight:bold }
-.chg .chg { color: #66f; }
-.head .head { color: #f4f; }
TOKENS
end
.pp { color:#579; }
.pt { color:#66f; font-weight:bold; }
.r { color:#5de; font-weight:bold; }
-.r, .kw { color:#5de; font-weight:bold }
-
-.ke { color: #808; }
.rx { background-color:#221133; }
.rx .k { color:#f8f; }
.ty { color:#339; font-weight:bold; }
.v { color:#036; }
.xt { color:#444; }
-
-.ins { background: #afa; }
-.del { background: #faa; }
-.chg { color: #aaf; background: #007; }
-.head { color: #f8f; background: #505 }
-
-.ins .ins { color: #080; font-weight:bold }
-.del .del { color: #800; font-weight:bold }
-.chg .chg { color: #66f; }
-.head .head { color: #f4f; }
TOKENS
end
h[k] = k.to_s
end
ClassOfKind.update with = {
- :annotation => 'at',
:attribute_name => 'an',
:attribute_name_fat => 'af',
:attribute_value => 'av',
:class_variable => 'cv',
:color => 'cr',
:comment => 'c',
- :complex => 'cm',
:constant => 'co',
:content => 'k',
- :decorator => 'de',
:definition => 'df',
:delimiter => 'dl',
:directive => 'di',
:doc => 'do',
- :doctype => 'dt',
:doc_string => 'ds',
:entity => 'en',
:error => 'er',
:function => 'fu',
:global_variable => 'gv',
:hex => 'hx',
- :imaginary => 'cm',
- :important => 'im',
:include => 'ic',
:inline => 'il',
:inline_delimiter => 'idl',
:instance_variable => 'iv',
:integer => 'i',
:interpreted => 'in',
- :keyword => 'kw',
- :key => 'ke',
:label => 'la',
:local_variable => 'lv',
:modifier => 'mod',
:pre_type => 'pt',
:predefined => 'pd',
:preprocessor => 'pp',
- :pseudo_class => 'ps',
:regexp => 'rx',
:reserved => 'r',
:shell => 'sh',
:tag_special => 'ts',
:type => 'ty',
:variable => 'v',
- :value => 'vl',
:xml_text => 'xt',
-
- :insert => 'ins',
- :delete => 'del',
- :change => 'chg',
- :head => 'head',
:ident => :NO_HIGHLIGHT, # 'id'
#:operator => 'op',
:space => :NO_HIGHLIGHT, # 'sp'
:plain => :NO_HIGHLIGHT,
}
- ClassOfKind[:method] = ClassOfKind[:function]
+ ClassOfKind[:procedure] = ClassOfKind[:method] = ClassOfKind[:function]
ClassOfKind[:open] = ClassOfKind[:close] = ClassOfKind[:delimiter]
ClassOfKind[:nesting_delimiter] = ClassOfKind[:delimiter]
ClassOfKind[:escape] = ClassOfKind[:delimiter]
#
# Tokens' subclass TokenStream allows streaming to save memory.
class Tokens < Array
-
- # The Scanner instance that created the tokens.
- attr_accessor :scanner
-
+
+ class << self
+
+ # Convert the token to a string.
+ #
+ # This format is used by Encoders.Tokens.
+ # It can be reverted using read_token.
+ def write_token text, type
+ if text.is_a? String
+ "#{type}\t#{escape(text)}\n"
+ else
+ ":#{text}\t#{type}\t\n"
+ end
+ end
+
+ # Read a token from the string.
+ #
+ # Inversion of write_token.
+ #
+ # TODO Test this!
+ def read_token token
+ type, text = token.split("\t", 2)
+ if type[0] == ?:
+ [text.to_sym, type[1..-1].to_sym]
+ else
+ [type.to_sym, unescape(text)]
+ end
+ end
+
+ # Escapes a string for use in write_token.
+ def escape text
+ text.gsub(/[\n\\]/, '\\\\\&')
+ end
+
+ # Unescapes a string created by escape.
+ def unescape text
+ text.gsub(/\\[\n\\]/) { |m| m[1,1] }
+ end
+
+ end
+
# Whether the object is a TokenStream.
#
# Returns false.
encode :text, options
end
+
# Redirects unknown methods to encoder calls.
#
# For example, if you call +tokens.html+, the HTML encoder
#
# TODO: Test this!
def fix
- tokens = self.class.new
# Check token nesting using a stack of kinds.
opened = []
- for type, kind in self
- case type
- when :open
- opened.push [:close, kind]
- when :begin_line
- opened.push [:end_line, kind]
- when :close, :end_line
+ for token, kind in self
+ if token == :open
+ opened.push kind
+ elsif token == :close
expected = opened.pop
- if [type, kind] != expected
+ if kind != expected
# Unexpected :close; decide what to do based on the kind:
- # - token was never opened: delete the :close (just skip it)
- next unless opened.rindex expected
# - token was opened earlier: also close tokens in between
- tokens << token until (token = opened.pop) == expected
+ # - token was never opened: delete the :close (skip with next)
+ next unless opened.rindex expected
+ tokens << [:close, kind] until (kind = opened.pop) == expected
end
end
- tokens << [type, kind]
+ tokens << [token, kind]
end
# Close remaining opened tokens
- tokens << token while token = opened.pop
+ tokens << [:close, kind] while kind = opened.pop
tokens
end
replace fix
end
- # TODO: Scanner#split_into_lines
- #
# Makes sure that:
# - newlines are single tokens
# (which means all other token are single-line)
#
# Returns self.
def << token
- @callback.call(*token)
+ @callback.call token
@size += 1
self
end
end
-end
+
+ # Token name abbreviations
+ require 'coderay/token_classes'
-if $0 == __FILE__
- $VERBOSE = true
- $: << File.join(File.dirname(__FILE__), '..')
- eval DATA.read, nil, $0, __LINE__ + 4
end
-
-__END__
-require 'test/unit'
-
-class TokensTest < Test::Unit::TestCase
-
- def test_creation
- assert CodeRay::Tokens < Array
- tokens = nil
- assert_nothing_raised do
- tokens = CodeRay::Tokens.new
- end
- assert_kind_of Array, tokens
- end
-
- def test_adding_tokens
- tokens = CodeRay::Tokens.new
- assert_nothing_raised do
- tokens << ['string', :type]
- tokens << ['()', :operator]
- end
- assert_equal tokens.size, 2
- end
-
- def test_dump_undump
- tokens = CodeRay::Tokens.new
- assert_nothing_raised do
- tokens << ['string', :type]
- tokens << ['()', :operator]
- end
- tokens2 = nil
- assert_nothing_raised do
- tokens2 = tokens.dump.undump
- end
- assert_equal tokens, tokens2
- end
-
-end
\ No newline at end of file
+++ /dev/null
-module CodeRay
-module Encoders
-
- load :token_class_filter
-
- class CommentFilter < TokenClassFilter
-
- register_for :comment_filter
-
- DEFAULT_OPTIONS = TokenClassFilter::DEFAULT_OPTIONS.merge \
- :exclude => [:comment]
-
- end
-
-end
-end
+++ /dev/null
-module CodeRay
-module Encoders
-
- class Filter < Encoder
-
- register_for :filter
-
- protected
- def setup options
- @out = Tokens.new
- end
-
- end
-
-end
-end
+++ /dev/null
-module CodeRay
-module Encoders
-
- # = JSON Encoder
- class JSON < Encoder
-
- register_for :json
- FILE_EXTENSION = 'json'
-
- protected
- def compile tokens, options
- require 'json'
- @out = tokens.to_a.to_json
- end
-
- end
-
-end
-end
+++ /dev/null
-module CodeRay
-module Encoders
-
- # Counts the LoC (Lines of Code). Returns an Integer >= 0.
- #
- # Everything that is not comment, markup, doctype/shebang, or an empty line,
- # is considered to be code.
- #
- # For example,
- # * HTML files not containing JavaScript have 0 LoC
- # * in a Java class without comments, LoC is the number of non-empty lines
- #
- # A Scanner class should define the token kinds that are not code in the
- # KINDS_NOT_LOC constant.
- class LinesOfCode < Encoder
-
- register_for :lines_of_code
-
- NON_EMPTY_LINE = /^\s*\S.*$/
-
- def compile tokens, options
- kinds_not_loc = tokens.scanner.class::KINDS_NOT_LOC
- code = tokens.token_class_filter :exclude => kinds_not_loc
- @loc = code.text.scan(NON_EMPTY_LINE).size
- end
-
- def finish options
- @loc
- end
-
- end
-
-end
-end
+++ /dev/null
-module CodeRay
-module Encoders
-
- load :filter
-
- class TokenClassFilter < Filter
-
- include Streamable
- register_for :token_class_filter
-
- DEFAULT_OPTIONS = {
- :exclude => [],
- :include => :all
- }
-
- protected
- def setup options
- super
- @exclude = options[:exclude]
- @include = options[:include]
- end
-
- def text_token text, kind
- [text, kind] if \
- (@include == :all || @include.include?(kind)) &&
- !(@exclude == :all || @exclude.include?(kind))
- end
-
- end
-
-end
-end
+++ /dev/null
-module CodeRay
-
- # A little hack to enable CodeRay highlighting in RedCloth.
- #
- # Usage:
- # require 'coderay'
- # require 'coderay/for_redcloth'
- # RedCloth.new('@[ruby]puts "Hello, World!"@').to_html
- #
- # Make sure you have RedCloth 4.0.3 activated, for example by calling
- # require 'rubygems'
- # before RedCloth is loaded and before calling CodeRay.for_redcloth.
- module ForRedCloth
-
- def self.install
- gem 'RedCloth', '>= 4.0.3' rescue nil
- require 'redcloth'
- unless RedCloth::VERSION.to_s >= '4.0.3'
- raise 'CodeRay.for_redcloth needs RedCloth version 4.0.3 or later.'
- end
- RedCloth::TextileDoc.send :include, ForRedCloth::TextileDoc
- RedCloth::Formatters::HTML.module_eval do
- def unescape(html)
- replacements = {
- '&' => '&',
- '"' => '"',
- '>' => '>',
- '<' => '<',
- }
- html.gsub(/&(?:amp|quot|[gl]t);/) { |entity| replacements[entity] }
- end
- undef code, bc_open, bc_close, escape_pre
- def code(opts) # :nodoc:
- opts[:block] = true
- if !opts[:lang] && RedCloth::VERSION.to_s >= '4.2.0'
- # simulating pre-4.2 behavior
- if opts[:text].sub!(/\A\[(\w+)\]/, '')
- if CodeRay::Scanners[$1].plugin_id == 'plaintext'
- opts[:text] = $& + opts[:text]
- else
- opts[:lang] = $1
- end
- end
- end
- if opts[:lang] && !filter_coderay
- require 'coderay'
- @in_bc ||= nil
- format = @in_bc ? :div : :span
- opts[:text] = unescape(opts[:text]) unless @in_bc
- highlighted_code = CodeRay.encode opts[:text], opts[:lang], format, :stream => true
- highlighted_code.sub!(/\A<(span|div)/) { |m| m + pba(@in_bc || opts) }
- highlighted_code
- else
- "<code#{pba(opts)}>#{opts[:text]}</code>"
- end
- end
- def bc_open(opts) # :nodoc:
- opts[:block] = true
- @in_bc = opts
- opts[:lang] ? '' : "<pre#{pba(opts)}>"
- end
- def bc_close(opts) # :nodoc:
- opts = @in_bc
- @in_bc = nil
- opts[:lang] ? '' : "</pre>\n"
- end
- def escape_pre(text)
- if @in_bc ||= nil
- text
- else
- html_esc(text, :html_escape_preformatted)
- end
- end
- end
- end
-
- module TextileDoc # :nodoc:
- attr_accessor :filter_coderay
- end
-
- end
-
-end
-
-CodeRay::ForRedCloth.install
\ No newline at end of file
+++ /dev/null
-module CodeRay
-module Scanners
-
- map \
- :h => :c,
- :cplusplus => :cpp,
- :'c++' => :cpp,
- :ecma => :java_script,
- :ecmascript => :java_script,
- :ecma_script => :java_script,
- :irb => :ruby,
- :javascript => :java_script,
- :js => :java_script,
- :nitro => :nitro_xhtml,
- :pascal => :delphi,
- :plain => :plaintext,
- :xhtml => :html,
- :yml => :yaml
-
- default :plain
-
-end
-end
+++ /dev/null
-module CodeRay
-module Scanners
-
- class CPlusPlus < Scanner
-
- include Streamable
-
- register_for :cpp
- file_extension 'cpp'
- title 'C++'
-
- # http://www.cppreference.com/wiki/keywords/start
- RESERVED_WORDS = [
- 'and', 'and_eq', 'asm', 'bitand', 'bitor', 'break',
- 'case', 'catch', 'class', 'compl', 'const_cast',
- 'continue', 'default', 'delete', 'do', 'dynamic_cast', 'else',
- 'enum', 'export', 'for', 'goto', 'if', 'namespace', 'new',
- 'not', 'not_eq', 'or', 'or_eq', 'reinterpret_cast', 'return',
- 'sizeof', 'static_cast', 'struct', 'switch', 'template',
- 'throw', 'try', 'typedef', 'typeid', 'typename', 'union',
- 'while', 'xor', 'xor_eq'
- ]
-
- PREDEFINED_TYPES = [
- 'bool', 'char', 'double', 'float', 'int', 'long',
- 'short', 'signed', 'unsigned', 'wchar_t', 'string'
- ]
- PREDEFINED_CONSTANTS = [
- 'false', 'true',
- 'EOF', 'NULL',
- ]
- PREDEFINED_VARIABLES = [
- 'this'
- ]
- DIRECTIVES = [
- 'auto', 'const', 'explicit', 'extern', 'friend', 'inline', 'mutable', 'operator',
- 'private', 'protected', 'public', 'register', 'static', 'using', 'virtual', 'void',
- 'volatile'
- ]
-
- IDENT_KIND = WordList.new(:ident).
- add(RESERVED_WORDS, :reserved).
- add(PREDEFINED_TYPES, :pre_type).
- add(PREDEFINED_VARIABLES, :local_variable).
- add(DIRECTIVES, :directive).
- add(PREDEFINED_CONSTANTS, :pre_constant)
-
- ESCAPE = / [rbfntv\n\\'"] | x[a-fA-F0-9]{1,2} | [0-7]{1,3} /x
- UNICODE_ESCAPE = / u[a-fA-F0-9]{4} | U[a-fA-F0-9]{8} /x
-
- def scan_tokens tokens, options
-
- state = :initial
-
- until eos?
-
- kind = nil
- match = nil
-
- case state
-
- when :initial
-
- if scan(/ \s+ | \\\n /x)
- kind = :space
-
- elsif scan(%r! // [^\n\\]* (?: \\. [^\n\\]* )* | /\* (?: .*? \*/ | .* ) !mx)
- kind = :comment
-
- elsif match = scan(/ \# \s* if \s* 0 /x)
- match << scan_until(/ ^\# (?:elif|else|endif) .*? $ | \z /xm) unless eos?
- kind = :comment
-
- elsif scan(/ [-+*=<>?:;,!&^|()\[\]{}~%]+ | \/=? | \.(?!\d) /x)
- kind = :operator
-
- elsif match = scan(/ [A-Za-z_][A-Za-z_0-9]* /x)
- kind = IDENT_KIND[match]
- if kind == :ident and check(/:(?!:)/)
- # FIXME: don't match a?b:c
- kind = :label
- elsif match == 'class'
- state = :class_name_expected
- end
-
- elsif scan(/\$/)
- kind = :ident
-
- elsif match = scan(/L?"/)
- tokens << [:open, :string]
- if match[0] == ?L
- tokens << ['L', :modifier]
- match = '"'
- end
- state = :string
- kind = :delimiter
-
- elsif scan(/#\s*(\w*)/)
- kind = :preprocessor
- state = :include_expected if self[1] == 'include'
-
- elsif scan(/ L?' (?: [^\'\n\\] | \\ #{ESCAPE} )? '? /ox)
- kind = :char
-
- elsif scan(/0[xX][0-9A-Fa-f]+/)
- kind = :hex
-
- elsif scan(/(?:0[0-7]+)(?![89.eEfF])/)
- kind = :oct
-
- elsif scan(/(?:\d+)(?![.eEfF])L?L?/)
- kind = :integer
-
- elsif scan(/\d[fF]?|\d*\.\d+(?:[eE][+-]?\d+)?[fF]?|\d+[eE][+-]?\d+[fF]?/)
- kind = :float
-
- else
- getch
- kind = :error
-
- end
-
- when :string
- if scan(/[^\\"]+/)
- kind = :content
- elsif scan(/"/)
- tokens << ['"', :delimiter]
- tokens << [:close, :string]
- state = :initial
- next
- elsif scan(/ \\ (?: #{ESCAPE} | #{UNICODE_ESCAPE} ) /mox)
- kind = :char
- elsif scan(/ \\ | $ /x)
- tokens << [:close, :string]
- kind = :error
- state = :initial
- else
- raise_inspect "else case \" reached; %p not handled." % peek(1), tokens
- end
-
- when :include_expected
- if scan(/<[^>\n]+>?|"[^"\n\\]*(?:\\.[^"\n\\]*)*"?/)
- kind = :include
- state = :initial
-
- elsif match = scan(/\s+/)
- kind = :space
- state = :initial if match.index ?\n
-
- else
- state = :initial
- next
-
- end
-
- when :class_name_expected
- if scan(/ [A-Za-z_][A-Za-z_0-9]* /x)
- kind = :class
- state = :initial
-
- elsif match = scan(/\s+/)
- kind = :space
-
- else
- getch
- kind = :error
- state = :initial
-
- end
-
- else
- raise_inspect 'Unknown state', tokens
-
- end
-
- match ||= matched
- if $DEBUG and not kind
- raise_inspect 'Error token %p in line %d' %
- [[match, kind], line], tokens
- end
- raise_inspect 'Empty token', tokens unless match
-
- tokens << [match, kind]
-
- end
-
- if state == :string
- tokens << [:close, :string]
- end
-
- tokens
- end
-
- end
-
-end
-end
+++ /dev/null
-module CodeRay
-module Scanners
-
- class CSS < Scanner
-
- register_for :css
-
- KINDS_NOT_LOC = [
- :comment,
- :class, :pseudo_class, :type,
- :constant, :directive,
- :key, :value, :operator, :color, :float,
- :error, :important,
- ]
-
- module RE
- NonASCII = /[\x80-\xFF]/
- Hex = /[0-9a-fA-F]/
- Unicode = /\\#{Hex}{1,6}(?:\r\n|\s)?/ # differs from standard because it allows uppercase hex too
- Escape = /#{Unicode}|\\[^\r\n\f0-9a-fA-F]/
- NMChar = /[-_a-zA-Z0-9]|#{NonASCII}|#{Escape}/
- NMStart = /[_a-zA-Z]|#{NonASCII}|#{Escape}/
- NL = /\r\n|\r|\n|\f/
- String1 = /"(?:[^\n\r\f\\"]|\\#{NL}|#{Escape})*"?/ # FIXME: buggy regexp
- String2 = /'(?:[^\n\r\f\\']|\\#{NL}|#{Escape})*'?/ # FIXME: buggy regexp
- String = /#{String1}|#{String2}/
-
- HexColor = /#(?:#{Hex}{6}|#{Hex}{3})/
- Color = /#{HexColor}/
-
- Num = /-?(?:[0-9]+|[0-9]*\.[0-9]+)/
- Name = /#{NMChar}+/
- Ident = /-?#{NMStart}#{NMChar}*/
- AtKeyword = /@#{Ident}/
- Percentage = /#{Num}%/
-
- reldimensions = %w[em ex px]
- absdimensions = %w[in cm mm pt pc]
- Unit = Regexp.union(*(reldimensions + absdimensions))
-
- Dimension = /#{Num}#{Unit}/
-
- Comment = %r! /\* (?: .*? \*/ | .* ) !mx
- Function = /(?:url|alpha)\((?:[^)\n\r\f]|\\\))*\)?/
-
- Id = /##{Name}/
- Class = /\.#{Name}/
- PseudoClass = /:#{Name}/
- AttributeSelector = /\[[^\]]*\]?/
-
- end
-
- def scan_tokens tokens, options
-
- value_expected = nil
- states = [:initial]
-
- until eos?
-
- kind = nil
- match = nil
-
- if scan(/\s+/)
- kind = :space
-
- elsif case states.last
- when :initial, :media
- if scan(/(?>#{RE::Ident})(?!\()|\*/ox)
- kind = :type
- elsif scan RE::Class
- kind = :class
- elsif scan RE::Id
- kind = :constant
- elsif scan RE::PseudoClass
- kind = :pseudo_class
- elsif match = scan(RE::AttributeSelector)
- # TODO: Improve highlighting inside of attribute selectors.
- tokens << [:open, :string]
- tokens << [match[0,1], :delimiter]
- tokens << [match[1..-2], :content] if match.size > 2
- tokens << [match[-1,1], :delimiter] if match[-1] == ?]
- tokens << [:close, :string]
- next
- elsif match = scan(/@media/)
- kind = :directive
- states.push :media_before_name
- end
-
- when :block
- if scan(/(?>#{RE::Ident})(?!\()/ox)
- if value_expected
- kind = :value
- else
- kind = :key
- end
- end
-
- when :media_before_name
- if scan RE::Ident
- kind = :type
- states[-1] = :media_after_name
- end
-
- when :media_after_name
- if scan(/\{/)
- kind = :operator
- states[-1] = :media
- end
-
- when :comment
- if scan(/(?:[^*\s]|\*(?!\/))+/)
- kind = :comment
- elsif scan(/\*\//)
- kind = :comment
- states.pop
- elsif scan(/\s+/)
- kind = :space
- end
-
- else
- raise_inspect 'Unknown state', tokens
-
- end
-
- elsif scan(/\/\*/)
- kind = :comment
- states.push :comment
-
- elsif scan(/\{/)
- value_expected = false
- kind = :operator
- states.push :block
-
- elsif scan(/\}/)
- value_expected = false
- if states.last == :block || states.last == :media
- kind = :operator
- states.pop
- else
- kind = :error
- end
-
- elsif match = scan(/#{RE::String}/o)
- tokens << [:open, :string]
- tokens << [match[0, 1], :delimiter]
- tokens << [match[1..-2], :content] if match.size > 2
- tokens << [match[-1, 1], :delimiter] if match.size >= 2
- tokens << [:close, :string]
- next
-
- elsif match = scan(/#{RE::Function}/o)
- tokens << [:open, :string]
- start = match[/^\w+\(/]
- tokens << [start, :delimiter]
- if match[-1] == ?)
- tokens << [match[start.size..-2], :content]
- tokens << [')', :delimiter]
- else
- tokens << [match[start.size..-1], :content]
- end
- tokens << [:close, :string]
- next
-
- elsif scan(/(?: #{RE::Dimension} | #{RE::Percentage} | #{RE::Num} )/ox)
- kind = :float
-
- elsif scan(/#{RE::Color}/o)
- kind = :color
-
- elsif scan(/! *important/)
- kind = :important
-
- elsif scan(/rgb\([^()\n]*\)?/)
- kind = :color
-
- elsif scan(/#{RE::AtKeyword}/o)
- kind = :directive
-
- elsif match = scan(/ [+>:;,.=()\/] /x)
- if match == ':'
- value_expected = true
- elsif match == ';'
- value_expected = false
- end
- kind = :operator
-
- else
- getch
- kind = :error
-
- end
-
- match ||= matched
- if $DEBUG and not kind
- raise_inspect 'Error token %p in line %d' %
- [[match, kind], line], tokens
- end
- raise_inspect 'Empty token', tokens unless match
-
- tokens << [match, kind]
-
- end
-
- tokens
- end
-
- end
-
-end
-end
+++ /dev/null
-module CodeRay
-module Scanners
-
- class Diff < Scanner
-
- register_for :diff
- title 'diff output'
-
- def scan_tokens tokens, options
-
- line_kind = nil
- state = :initial
-
- until eos?
- kind = match = nil
-
- if match = scan(/\n/)
- if line_kind
- tokens << [:end_line, line_kind]
- line_kind = nil
- end
- tokens << [match, :space]
- next
- end
-
- case state
-
- when :initial
- if match = scan(/--- |\+\+\+ |=+|_+/)
- tokens << [:begin_line, line_kind = :head]
- tokens << [match, :head]
- next unless match = scan(/.+/)
- kind = :plain
- elsif match = scan(/Index: |Property changes on: /)
- tokens << [:begin_line, line_kind = :head]
- tokens << [match, :head]
- next unless match = scan(/.+/)
- kind = :plain
- elsif match = scan(/Added: /)
- tokens << [:begin_line, line_kind = :head]
- tokens << [match, :head]
- next unless match = scan(/.+/)
- kind = :plain
- state = :added
- elsif match = scan(/\\ /)
- tokens << [:begin_line, line_kind = :change]
- tokens << [match, :change]
- next unless match = scan(/.+/)
- kind = :plain
- elsif scan(/(@@)((?>[^@\n]*))(@@)/)
- tokens << [:begin_line, line_kind = :change]
- tokens << [self[1], :change]
- tokens << [self[2], :plain]
- tokens << [self[3], :change]
- next unless match = scan(/.+/)
- kind = :plain
- elsif match = scan(/\+/)
- tokens << [:begin_line, line_kind = :insert]
- tokens << [match, :insert]
- next unless match = scan(/.+/)
- kind = :plain
- elsif match = scan(/-/)
- tokens << [:begin_line, line_kind = :delete]
- tokens << [match, :delete]
- next unless match = scan(/.+/)
- kind = :plain
- elsif scan(/ .*/)
- kind = :comment
- elsif scan(/.+/)
- tokens << [:begin_line, line_kind = :head]
- kind = :plain
- else
- raise_inspect 'else case rached'
- end
-
- when :added
- if match = scan(/ \+/)
- tokens << [:begin_line, line_kind = :insert]
- tokens << [match, :insert]
- next unless match = scan(/.+/)
- kind = :plain
- else
- state = :initial
- next
- end
- end
-
- match ||= matched
- if $DEBUG and not kind
- raise_inspect 'Error token %p in line %d' %
- [[match, kind], line], tokens
- end
- raise_inspect 'Empty token', tokens unless match
-
- tokens << [match, kind]
- end
-
- tokens << [:end_line, line_kind] if line_kind
- tokens
- end
-
- end
-
-end
-end
+++ /dev/null
-module CodeRay
-module Scanners
-
- load :java
-
- class Groovy < Java
-
- include Streamable
- register_for :groovy
-
- # TODO: Check this!
- GROOVY_KEYWORDS = %w[
- as assert def in
- ]
- KEYWORDS_EXPECTING_VALUE = WordList.new.add %w[
- case instanceof new return throw typeof while as assert in
- ]
- GROOVY_MAGIC_VARIABLES = %w[ it ]
-
- IDENT_KIND = Java::IDENT_KIND.dup.
- add(GROOVY_KEYWORDS, :keyword).
- add(GROOVY_MAGIC_VARIABLES, :local_variable)
-
- ESCAPE = / [bfnrtv$\n\\'"] | x[a-fA-F0-9]{1,2} | [0-7]{1,3} /x
- UNICODE_ESCAPE = / u[a-fA-F0-9]{4} /x # no 4-byte unicode chars? U[a-fA-F0-9]{8}
- REGEXP_ESCAPE = / [bfnrtv\n\\'"] | x[a-fA-F0-9]{1,2} | [0-7]{1,3} | \d | [bBdDsSwW\/] /x
-
- # TODO: interpretation inside ', ", /
- STRING_CONTENT_PATTERN = {
- "'" => /(?>\\[^\\'\n]+|[^\\'\n]+)+/,
- '"' => /[^\\$"\n]+/,
- "'''" => /(?>[^\\']+|'(?!''))+/,
- '"""' => /(?>[^\\$"]+|"(?!""))+/,
- '/' => /[^\\$\/\n]+/,
- }
-
- def scan_tokens tokens, options
-
- state = :initial
- inline_block_stack = []
- inline_block_paren_depth = nil
- string_delimiter = nil
- import_clause = class_name_follows = last_token = after_def = false
- value_expected = true
-
- until eos?
-
- kind = nil
- match = nil
-
- case state
-
- when :initial
-
- if match = scan(/ \s+ | \\\n /x)
- tokens << [match, :space]
- if match.index ?\n
- import_clause = after_def = false
- value_expected = true unless value_expected
- end
- next
-
- elsif scan(%r! // [^\n\\]* (?: \\. [^\n\\]* )* | /\* (?: .*? \*/ | .* ) !mx)
- value_expected = true
- after_def = false
- kind = :comment
-
- elsif bol? && scan(/ \#!.* /x)
- kind = :doctype
-
- elsif import_clause && scan(/ (?!as) #{IDENT} (?: \. #{IDENT} )* (?: \.\* )? /ox)
- after_def = value_expected = false
- kind = :include
-
- elsif match = scan(/ #{IDENT} | \[\] /ox)
- kind = IDENT_KIND[match]
- value_expected = (kind == :keyword) && KEYWORDS_EXPECTING_VALUE[match]
- if last_token == '.'
- kind = :ident
- elsif class_name_follows
- kind = :class
- class_name_follows = false
- elsif after_def && check(/\s*[({]/)
- kind = :method
- after_def = false
- elsif kind == :ident && last_token != '?' && check(/:/)
- kind = :key
- else
- class_name_follows = true if match == 'class' || (import_clause && match == 'as')
- import_clause = match == 'import'
- after_def = true if match == 'def'
- end
-
- elsif scan(/;/)
- import_clause = after_def = false
- value_expected = true
- kind = :operator
-
- elsif scan(/\{/)
- class_name_follows = after_def = false
- value_expected = true
- kind = :operator
- if !inline_block_stack.empty?
- inline_block_paren_depth += 1
- end
-
- # TODO: ~'...', ~"..." and ~/.../ style regexps
- elsif match = scan(/ \.\.<? | \*?\.(?!\d)@? | \.& | \?:? | [,?:(\[] | -[->] | \+\+ |
- && | \|\| | \*\*=? | ==?~ | <=?>? | [-+*%^~&|>=!]=? | <<<?=? | >>>?=? /x)
- value_expected = true
- value_expected = :regexp if match == '~'
- after_def = false
- kind = :operator
-
- elsif match = scan(/ [)\]}] /x)
- value_expected = after_def = false
- if !inline_block_stack.empty? && match == '}'
- inline_block_paren_depth -= 1
- if inline_block_paren_depth == 0 # closing brace of inline block reached
- tokens << [match, :inline_delimiter]
- tokens << [:close, :inline]
- state, string_delimiter, inline_block_paren_depth = inline_block_stack.pop
- next
- end
- end
-
- elsif check(/[\d.]/)
- after_def = value_expected = false
- if scan(/0[xX][0-9A-Fa-f]+/)
- kind = :hex
- elsif scan(/(?>0[0-7]+)(?![89.eEfF])/)
- kind = :oct
- elsif scan(/\d+[fFdD]|\d*\.\d+(?:[eE][+-]?\d+)?[fFdD]?|\d+[eE][+-]?\d+[fFdD]?/)
- kind = :float
- elsif scan(/\d+[lLgG]?/)
- kind = :integer
- end
-
- elsif match = scan(/'''|"""/)
- after_def = value_expected = false
- state = :multiline_string
- tokens << [:open, :string]
- string_delimiter = match
- kind = :delimiter
-
- # TODO: record.'name'
- elsif match = scan(/["']/)
- after_def = value_expected = false
- state = match == '/' ? :regexp : :string
- tokens << [:open, state]
- string_delimiter = match
- kind = :delimiter
-
- elsif value_expected && (match = scan(/\//))
- after_def = value_expected = false
- tokens << [:open, :regexp]
- state = :regexp
- string_delimiter = '/'
- kind = :delimiter
-
- elsif scan(/ @ #{IDENT} /ox)
- after_def = value_expected = false
- kind = :annotation
-
- elsif scan(/\//)
- after_def = false
- value_expected = true
- kind = :operator
-
- else
- getch
- kind = :error
-
- end
-
- when :string, :regexp, :multiline_string
- if scan(STRING_CONTENT_PATTERN[string_delimiter])
- kind = :content
-
- elsif match = scan(state == :multiline_string ? /'''|"""/ : /["'\/]/)
- tokens << [match, :delimiter]
- if state == :regexp
- # TODO: regexp modifiers? s, m, x, i?
- modifiers = scan(/[ix]+/)
- tokens << [modifiers, :modifier] if modifiers && !modifiers.empty?
- end
- state = :string if state == :multiline_string
- tokens << [:close, state]
- string_delimiter = nil
- after_def = value_expected = false
- state = :initial
- next
-
- elsif (state == :string || state == :multiline_string) &&
- (match = scan(/ \\ (?: #{ESCAPE} | #{UNICODE_ESCAPE} ) /mox))
- if string_delimiter[0] == ?' && !(match == "\\\\" || match == "\\'")
- kind = :content
- else
- kind = :char
- end
- elsif state == :regexp && scan(/ \\ (?: #{REGEXP_ESCAPE} | #{UNICODE_ESCAPE} ) /mox)
- kind = :char
-
- elsif match = scan(/ \$ #{IDENT} /mox)
- tokens << [:open, :inline]
- tokens << ['$', :inline_delimiter]
- match = match[1..-1]
- tokens << [match, IDENT_KIND[match]]
- tokens << [:close, :inline]
- next
- elsif match = scan(/ \$ \{ /x)
- tokens << [:open, :inline]
- tokens << ['${', :inline_delimiter]
- inline_block_stack << [state, string_delimiter, inline_block_paren_depth]
- inline_block_paren_depth = 1
- state = :initial
- next
-
- elsif scan(/ \$ /mx)
- kind = :content
-
- elsif scan(/ \\. /mx)
- kind = :content
-
- elsif scan(/ \\ | \n /x)
- tokens << [:close, state]
- kind = :error
- after_def = value_expected = false
- state = :initial
-
- else
- raise_inspect "else case \" reached; %p not handled." % peek(1), tokens
- end
-
- else
- raise_inspect 'Unknown state', tokens
-
- end
-
- match ||= matched
- if $DEBUG and not kind
- raise_inspect 'Error token %p in line %d' %
- [[match, kind], line], tokens
- end
- raise_inspect 'Empty token', tokens unless match
-
- last_token = match unless [:space, :comment, :doctype].include? kind
-
- tokens << [match, kind]
-
- end
-
- if [:multiline_string, :string, :regexp].include? state
- tokens << [:close, state]
- end
-
- tokens
- end
-
- end
-
-end
-end
+++ /dev/null
-module CodeRay
-module Scanners
-
- class Java < Scanner
-
- include Streamable
- register_for :java
- helper :builtin_types
-
- # http://java.sun.com/docs/books/tutorial/java/nutsandbolts/_keywords.html
- KEYWORDS = %w[
- assert break case catch continue default do else
- finally for if instanceof import new package
- return switch throw try typeof while
- debugger export
- ]
- RESERVED = %w[ const goto ]
- CONSTANTS = %w[ false null true ]
- MAGIC_VARIABLES = %w[ this super ]
- TYPES = %w[
- boolean byte char class double enum float int interface long
- short void
- ] << '[]' # String[] should be highlighted as a type
- DIRECTIVES = %w[
- abstract extends final implements native private protected public
- static strictfp synchronized throws transient volatile
- ]
-
- IDENT_KIND = WordList.new(:ident).
- add(KEYWORDS, :keyword).
- add(RESERVED, :reserved).
- add(CONSTANTS, :pre_constant).
- add(MAGIC_VARIABLES, :local_variable).
- add(TYPES, :type).
- add(BuiltinTypes::List, :pre_type).
- add(BuiltinTypes::List.select { |builtin| builtin[/(Error|Exception)$/] }, :exception).
- add(DIRECTIVES, :directive)
-
- ESCAPE = / [bfnrtv\n\\'"] | x[a-fA-F0-9]{1,2} | [0-7]{1,3} /x
- UNICODE_ESCAPE = / u[a-fA-F0-9]{4} | U[a-fA-F0-9]{8} /x
- STRING_CONTENT_PATTERN = {
- "'" => /[^\\']+/,
- '"' => /[^\\"]+/,
- '/' => /[^\\\/]+/,
- }
- IDENT = /[a-zA-Z_][A-Za-z_0-9]*/
-
- def scan_tokens tokens, options
-
- state = :initial
- string_delimiter = nil
- import_clause = class_name_follows = last_token_dot = false
-
- until eos?
-
- kind = nil
- match = nil
-
- case state
-
- when :initial
-
- if match = scan(/ \s+ | \\\n /x)
- tokens << [match, :space]
- next
-
- elsif match = scan(%r! // [^\n\\]* (?: \\. [^\n\\]* )* | /\* (?: .*? \*/ | .* ) !mx)
- tokens << [match, :comment]
- next
-
- elsif import_clause && scan(/ #{IDENT} (?: \. #{IDENT} )* /ox)
- kind = :include
-
- elsif match = scan(/ #{IDENT} | \[\] /ox)
- kind = IDENT_KIND[match]
- if last_token_dot
- kind = :ident
- elsif class_name_follows
- kind = :class
- class_name_follows = false
- else
- import_clause = true if match == 'import'
- class_name_follows = true if match == 'class' || match == 'interface'
- end
-
- elsif scan(/ \.(?!\d) | [,?:()\[\]}] | -- | \+\+ | && | \|\| | \*\*=? | [-+*\/%^~&|<>=!]=? | <<<?=? | >>>?=? /x)
- kind = :operator
-
- elsif scan(/;/)
- import_clause = false
- kind = :operator
-
- elsif scan(/\{/)
- class_name_follows = false
- kind = :operator
-
- elsif check(/[\d.]/)
- if scan(/0[xX][0-9A-Fa-f]+/)
- kind = :hex
- elsif scan(/(?>0[0-7]+)(?![89.eEfF])/)
- kind = :oct
- elsif scan(/\d+[fFdD]|\d*\.\d+(?:[eE][+-]?\d+)?[fFdD]?|\d+[eE][+-]?\d+[fFdD]?/)
- kind = :float
- elsif scan(/\d+[lL]?/)
- kind = :integer
- end
-
- elsif match = scan(/["']/)
- tokens << [:open, :string]
- state = :string
- string_delimiter = match
- kind = :delimiter
-
- elsif scan(/ @ #{IDENT} /ox)
- kind = :annotation
-
- else
- getch
- kind = :error
-
- end
-
- when :string
- if scan(STRING_CONTENT_PATTERN[string_delimiter])
- kind = :content
- elsif match = scan(/["'\/]/)
- tokens << [match, :delimiter]
- tokens << [:close, state]
- string_delimiter = nil
- state = :initial
- next
- elsif state == :string && (match = scan(/ \\ (?: #{ESCAPE} | #{UNICODE_ESCAPE} ) /mox))
- if string_delimiter == "'" && !(match == "\\\\" || match == "\\'")
- kind = :content
- else
- kind = :char
- end
- elsif scan(/\\./m)
- kind = :content
- elsif scan(/ \\ | $ /x)
- tokens << [:close, :delimiter]
- kind = :error
- state = :initial
- else
- raise_inspect "else case \" reached; %p not handled." % peek(1), tokens
- end
-
- else
- raise_inspect 'Unknown state', tokens
-
- end
-
- match ||= matched
- if $DEBUG and not kind
- raise_inspect 'Error token %p in line %d' %
- [[match, kind], line], tokens
- end
- raise_inspect 'Empty token', tokens unless match
-
- last_token_dot = match == '.'
-
- tokens << [match, kind]
-
- end
-
- if state == :string
- tokens << [:close, state]
- end
-
- tokens
- end
-
- end
-
-end
-end
+++ /dev/null
-module CodeRay
-module Scanners
-
- module Java::BuiltinTypes # :nodoc:
-
- List = %w[
- AbstractAction AbstractBorder AbstractButton AbstractCellEditor AbstractCollection
- AbstractColorChooserPanel AbstractDocument AbstractExecutorService AbstractInterruptibleChannel
- AbstractLayoutCache AbstractList AbstractListModel AbstractMap AbstractMethodError AbstractPreferences
- AbstractQueue AbstractQueuedSynchronizer AbstractSelectableChannel AbstractSelectionKey AbstractSelector
- AbstractSequentialList AbstractSet AbstractSpinnerModel AbstractTableModel AbstractUndoableEdit
- AbstractWriter AccessControlContext AccessControlException AccessController AccessException Accessible
- AccessibleAction AccessibleAttributeSequence AccessibleBundle AccessibleComponent AccessibleContext
- AccessibleEditableText AccessibleExtendedComponent AccessibleExtendedTable AccessibleExtendedText
- AccessibleHyperlink AccessibleHypertext AccessibleIcon AccessibleKeyBinding AccessibleObject
- AccessibleRelation AccessibleRelationSet AccessibleResourceBundle AccessibleRole AccessibleSelection
- AccessibleState AccessibleStateSet AccessibleStreamable AccessibleTable AccessibleTableModelChange
- AccessibleText AccessibleTextSequence AccessibleValue AccountException AccountExpiredException
- AccountLockedException AccountNotFoundException Acl AclEntry AclNotFoundException Action ActionEvent
- ActionListener ActionMap ActionMapUIResource Activatable ActivateFailedException ActivationDesc
- ActivationException ActivationGroup ActivationGroupDesc ActivationGroupID ActivationGroup_Stub
- ActivationID ActivationInstantiator ActivationMonitor ActivationSystem Activator ActiveEvent
- ActivityCompletedException ActivityRequiredException Adjustable AdjustmentEvent AdjustmentListener
- Adler32 AffineTransform AffineTransformOp AlgorithmParameterGenerator AlgorithmParameterGeneratorSpi
- AlgorithmParameters AlgorithmParameterSpec AlgorithmParametersSpi AllPermission AlphaComposite
- AlreadyBoundException AlreadyConnectedException AncestorEvent AncestorListener AnnotatedElement
- Annotation AnnotationFormatError AnnotationTypeMismatchException AppConfigurationEntry Appendable Applet
- AppletContext AppletInitializer AppletStub Arc2D Area AreaAveragingScaleFilter ArithmeticException Array
- ArrayBlockingQueue ArrayIndexOutOfBoundsException ArrayList Arrays ArrayStoreException ArrayType
- AssertionError AsyncBoxView AsynchronousCloseException AtomicBoolean AtomicInteger AtomicIntegerArray
- AtomicIntegerFieldUpdater AtomicLong AtomicLongArray AtomicLongFieldUpdater AtomicMarkableReference
- AtomicReference AtomicReferenceArray AtomicReferenceFieldUpdater AtomicStampedReference Attribute
- AttributeChangeNotification AttributeChangeNotificationFilter AttributedCharacterIterator
- AttributedString AttributeException AttributeInUseException AttributeList AttributeModificationException
- AttributeNotFoundException Attributes AttributeSet AttributeSetUtilities AttributeValueExp AudioClip
- AudioFileFormat AudioFileReader AudioFileWriter AudioFormat AudioInputStream AudioPermission AudioSystem
- AuthenticationException AuthenticationNotSupportedException Authenticator AuthorizeCallback
- AuthPermission AuthProvider Autoscroll AWTError AWTEvent AWTEventListener AWTEventListenerProxy
- AWTEventMulticaster AWTException AWTKeyStroke AWTPermission BackingStoreException
- BadAttributeValueExpException BadBinaryOpValueExpException BadLocationException BadPaddingException
- BadStringOperationException BandCombineOp BandedSampleModel BaseRowSet BasicArrowButton BasicAttribute
- BasicAttributes BasicBorders BasicButtonListener BasicButtonUI BasicCheckBoxMenuItemUI BasicCheckBoxUI
- BasicColorChooserUI BasicComboBoxEditor BasicComboBoxRenderer BasicComboBoxUI BasicComboPopup
- BasicControl BasicDesktopIconUI BasicDesktopPaneUI BasicDirectoryModel BasicEditorPaneUI
- BasicFileChooserUI BasicFormattedTextFieldUI BasicGraphicsUtils BasicHTML BasicIconFactory
- BasicInternalFrameTitlePane BasicInternalFrameUI BasicLabelUI BasicListUI BasicLookAndFeel
- BasicMenuBarUI BasicMenuItemUI BasicMenuUI BasicOptionPaneUI BasicPanelUI BasicPasswordFieldUI
- BasicPermission BasicPopupMenuSeparatorUI BasicPopupMenuUI BasicProgressBarUI BasicRadioButtonMenuItemUI
- BasicRadioButtonUI BasicRootPaneUI BasicScrollBarUI BasicScrollPaneUI BasicSeparatorUI BasicSliderUI
- BasicSpinnerUI BasicSplitPaneDivider BasicSplitPaneUI BasicStroke BasicTabbedPaneUI BasicTableHeaderUI
- BasicTableUI BasicTextAreaUI BasicTextFieldUI BasicTextPaneUI BasicTextUI BasicToggleButtonUI
- BasicToolBarSeparatorUI BasicToolBarUI BasicToolTipUI BasicTreeUI BasicViewportUI BatchUpdateException
- BeanContext BeanContextChild BeanContextChildComponentProxy BeanContextChildSupport
- BeanContextContainerProxy BeanContextEvent BeanContextMembershipEvent BeanContextMembershipListener
- BeanContextProxy BeanContextServiceAvailableEvent BeanContextServiceProvider
- BeanContextServiceProviderBeanInfo BeanContextServiceRevokedEvent BeanContextServiceRevokedListener
- BeanContextServices BeanContextServicesListener BeanContextServicesSupport BeanContextSupport
- BeanDescriptor BeanInfo Beans BevelBorder Bidi BigDecimal BigInteger BinaryRefAddr BindException Binding
- BitSet Blob BlockingQueue BlockView BMPImageWriteParam Book Boolean BooleanControl Border BorderFactory
- BorderLayout BorderUIResource BoundedRangeModel Box BoxLayout BoxView BreakIterator
- BrokenBarrierException Buffer BufferCapabilities BufferedImage BufferedImageFilter BufferedImageOp
- BufferedInputStream BufferedOutputStream BufferedReader BufferedWriter BufferOverflowException
- BufferStrategy BufferUnderflowException Button ButtonGroup ButtonModel ButtonUI Byte
- ByteArrayInputStream ByteArrayOutputStream ByteBuffer ByteChannel ByteLookupTable ByteOrder CachedRowSet
- CacheRequest CacheResponse Calendar Callable CallableStatement Callback CallbackHandler
- CancelablePrintJob CancellationException CancelledKeyException CannotProceedException
- CannotRedoException CannotUndoException Canvas CardLayout Caret CaretEvent CaretListener CellEditor
- CellEditorListener CellRendererPane Certificate CertificateEncodingException CertificateException
- CertificateExpiredException CertificateFactory CertificateFactorySpi CertificateNotYetValidException
- CertificateParsingException CertPath CertPathBuilder CertPathBuilderException CertPathBuilderResult
- CertPathBuilderSpi CertPathParameters CertPathTrustManagerParameters CertPathValidator
- CertPathValidatorException CertPathValidatorResult CertPathValidatorSpi CertSelector CertStore
- CertStoreException CertStoreParameters CertStoreSpi ChangedCharSetException ChangeEvent ChangeListener
- Channel Channels Character CharacterCodingException CharacterIterator CharArrayReader CharArrayWriter
- CharBuffer CharConversionException CharSequence Charset CharsetDecoder CharsetEncoder CharsetProvider
- Checkbox CheckboxGroup CheckboxMenuItem CheckedInputStream CheckedOutputStream Checksum Choice
- ChoiceCallback ChoiceFormat Chromaticity Cipher CipherInputStream CipherOutputStream CipherSpi Class
- ClassCastException ClassCircularityError ClassDefinition ClassDesc ClassFileTransformer ClassFormatError
- ClassLoader ClassLoaderRepository ClassLoadingMXBean ClassNotFoundException Clip Clipboard
- ClipboardOwner Clob Cloneable CloneNotSupportedException Closeable ClosedByInterruptException
- ClosedChannelException ClosedSelectorException CMMException CoderMalfunctionError CoderResult CodeSigner
- CodeSource CodingErrorAction CollationElementIterator CollationKey Collator Collection
- CollectionCertStoreParameters Collections Color ColorChooserComponentFactory ColorChooserUI
- ColorConvertOp ColorModel ColorSelectionModel ColorSpace ColorSupported ColorType ColorUIResource
- ComboBoxEditor ComboBoxModel ComboBoxUI ComboPopup CommunicationException Comparable Comparator
- CompilationMXBean Compiler CompletionService Component ComponentAdapter ComponentColorModel
- ComponentEvent ComponentInputMap ComponentInputMapUIResource ComponentListener ComponentOrientation
- ComponentSampleModel ComponentUI ComponentView Composite CompositeContext CompositeData
- CompositeDataSupport CompositeName CompositeType CompositeView CompoundBorder CompoundControl
- CompoundEdit CompoundName Compression ConcurrentHashMap ConcurrentLinkedQueue ConcurrentMap
- ConcurrentModificationException Condition Configuration ConfigurationException ConfirmationCallback
- ConnectException ConnectIOException Connection ConnectionEvent ConnectionEventListener
- ConnectionPendingException ConnectionPoolDataSource ConsoleHandler Constructor Container
- ContainerAdapter ContainerEvent ContainerListener ContainerOrderFocusTraversalPolicy ContentHandler
- ContentHandlerFactory ContentModel Context ContextNotEmptyException ContextualRenderedImageFactory
- Control ControlFactory ControllerEventListener ConvolveOp CookieHandler Copies CopiesSupported
- CopyOnWriteArrayList CopyOnWriteArraySet CountDownLatch CounterMonitor CounterMonitorMBean CRC32
- CredentialException CredentialExpiredException CredentialNotFoundException CRL CRLException CRLSelector
- CropImageFilter CSS CubicCurve2D Currency Cursor Customizer CyclicBarrier DatabaseMetaData DataBuffer
- DataBufferByte DataBufferDouble DataBufferFloat DataBufferInt DataBufferShort DataBufferUShort
- DataFlavor DataFormatException DatagramChannel DatagramPacket DatagramSocket DatagramSocketImpl
- DatagramSocketImplFactory DataInput DataInputStream DataLine DataOutput DataOutputStream DataSource
- DataTruncation DatatypeConfigurationException DatatypeConstants DatatypeFactory Date DateFormat
- DateFormatSymbols DateFormatter DateTimeAtCompleted DateTimeAtCreation DateTimeAtProcessing
- DateTimeSyntax DebugGraphics DecimalFormat DecimalFormatSymbols DefaultBoundedRangeModel
- DefaultButtonModel DefaultCaret DefaultCellEditor DefaultColorSelectionModel DefaultComboBoxModel
- DefaultDesktopManager DefaultEditorKit DefaultFocusManager DefaultFocusTraversalPolicy DefaultFormatter
- DefaultFormatterFactory DefaultHighlighter DefaultKeyboardFocusManager DefaultListCellRenderer
- DefaultListModel DefaultListSelectionModel DefaultLoaderRepository DefaultMenuLayout DefaultMetalTheme
- DefaultMutableTreeNode DefaultPersistenceDelegate DefaultSingleSelectionModel DefaultStyledDocument
- DefaultTableCellRenderer DefaultTableColumnModel DefaultTableModel DefaultTextUI DefaultTreeCellEditor
- DefaultTreeCellRenderer DefaultTreeModel DefaultTreeSelectionModel Deflater DeflaterOutputStream Delayed
- DelayQueue DelegationPermission Deprecated Descriptor DescriptorAccess DescriptorSupport DESedeKeySpec
- DesignMode DESKeySpec DesktopIconUI DesktopManager DesktopPaneUI Destination Destroyable
- DestroyFailedException DGC DHGenParameterSpec DHKey DHParameterSpec DHPrivateKey DHPrivateKeySpec
- DHPublicKey DHPublicKeySpec Dialog Dictionary DigestException DigestInputStream DigestOutputStream
- Dimension Dimension2D DimensionUIResource DirContext DirectColorModel DirectoryManager DirObjectFactory
- DirStateFactory DisplayMode DnDConstants Doc DocAttribute DocAttributeSet DocFlavor DocPrintJob Document
- DocumentBuilder DocumentBuilderFactory Documented DocumentEvent DocumentFilter DocumentListener
- DocumentName DocumentParser DomainCombiner DOMLocator DOMResult DOMSource Double DoubleBuffer
- DragGestureEvent DragGestureListener DragGestureRecognizer DragSource DragSourceAdapter
- DragSourceContext DragSourceDragEvent DragSourceDropEvent DragSourceEvent DragSourceListener
- DragSourceMotionListener Driver DriverManager DriverPropertyInfo DropTarget DropTargetAdapter
- DropTargetContext DropTargetDragEvent DropTargetDropEvent DropTargetEvent DropTargetListener DSAKey
- DSAKeyPairGenerator DSAParameterSpec DSAParams DSAPrivateKey DSAPrivateKeySpec DSAPublicKey
- DSAPublicKeySpec DTD DTDConstants DuplicateFormatFlagsException Duration DynamicMBean ECField ECFieldF2m
- ECFieldFp ECGenParameterSpec ECKey ECParameterSpec ECPoint ECPrivateKey ECPrivateKeySpec ECPublicKey
- ECPublicKeySpec EditorKit Element ElementIterator ElementType Ellipse2D EllipticCurve EmptyBorder
- EmptyStackException EncodedKeySpec Encoder EncryptedPrivateKeyInfo Entity Enum
- EnumConstantNotPresentException EnumControl Enumeration EnumMap EnumSet EnumSyntax EOFException Error
- ErrorListener ErrorManager EtchedBorder Event EventContext EventDirContext EventHandler EventListener
- EventListenerList EventListenerProxy EventObject EventQueue EventSetDescriptor Exception
- ExceptionInInitializerError ExceptionListener Exchanger ExecutionException Executor
- ExecutorCompletionService Executors ExecutorService ExemptionMechanism ExemptionMechanismException
- ExemptionMechanismSpi ExpandVetoException ExportException Expression ExtendedRequest ExtendedResponse
- Externalizable FactoryConfigurationError FailedLoginException FeatureDescriptor Fidelity Field
- FieldPosition FieldView File FileCacheImageInputStream FileCacheImageOutputStream FileChannel
- FileChooserUI FileDescriptor FileDialog FileFilter FileHandler FileImageInputStream
- FileImageOutputStream FileInputStream FileLock FileLockInterruptionException FilenameFilter FileNameMap
- FileNotFoundException FileOutputStream FilePermission FileReader FileSystemView FileView FileWriter
- Filter FilteredImageSource FilteredRowSet FilterInputStream FilterOutputStream FilterReader FilterWriter
- Finishings FixedHeightLayoutCache FlatteningPathIterator FlavorEvent FlavorException FlavorListener
- FlavorMap FlavorTable Float FloatBuffer FloatControl FlowLayout FlowView Flushable FocusAdapter
- FocusEvent FocusListener FocusManager FocusTraversalPolicy Font FontFormatException FontMetrics
- FontRenderContext FontUIResource Format FormatConversionProvider FormatFlagsConversionMismatchException
- Formattable FormattableFlags Formatter FormatterClosedException FormSubmitEvent FormView Frame Future
- FutureTask GapContent GarbageCollectorMXBean GatheringByteChannel GaugeMonitor GaugeMonitorMBean
- GeneralPath GeneralSecurityException GenericArrayType GenericDeclaration GenericSignatureFormatError
- GlyphJustificationInfo GlyphMetrics GlyphVector GlyphView GradientPaint GraphicAttribute Graphics
- Graphics2D GraphicsConfigTemplate GraphicsConfiguration GraphicsDevice GraphicsEnvironment GrayFilter
- GregorianCalendar GridBagConstraints GridBagLayout GridLayout Group Guard GuardedObject GZIPInputStream
- GZIPOutputStream Handler HandshakeCompletedEvent HandshakeCompletedListener HasControls HashAttributeSet
- HashDocAttributeSet HashMap HashPrintJobAttributeSet HashPrintRequestAttributeSet
- HashPrintServiceAttributeSet HashSet Hashtable HeadlessException HierarchyBoundsAdapter
- HierarchyBoundsListener HierarchyEvent HierarchyListener Highlighter HostnameVerifier HTML HTMLDocument
- HTMLEditorKit HTMLFrameHyperlinkEvent HTMLWriter HttpRetryException HttpsURLConnection HttpURLConnection
- HyperlinkEvent HyperlinkListener ICC_ColorSpace ICC_Profile ICC_ProfileGray ICC_ProfileRGB Icon
- IconUIResource IconView Identity IdentityHashMap IdentityScope IIOByteBuffer IIOException IIOImage
- IIOInvalidTreeException IIOMetadata IIOMetadataController IIOMetadataFormat IIOMetadataFormatImpl
- IIOMetadataNode IIOParam IIOParamController IIOReadProgressListener IIOReadUpdateListener
- IIOReadWarningListener IIORegistry IIOServiceProvider IIOWriteProgressListener IIOWriteWarningListener
- IllegalAccessError IllegalAccessException IllegalArgumentException IllegalBlockingModeException
- IllegalBlockSizeException IllegalCharsetNameException IllegalClassFormatException
- IllegalComponentStateException IllegalFormatCodePointException IllegalFormatConversionException
- IllegalFormatException IllegalFormatFlagsException IllegalFormatPrecisionException
- IllegalFormatWidthException IllegalMonitorStateException IllegalPathStateException
- IllegalSelectorException IllegalStateException IllegalThreadStateException Image ImageCapabilities
- ImageConsumer ImageFilter ImageGraphicAttribute ImageIcon ImageInputStream ImageInputStreamImpl
- ImageInputStreamSpi ImageIO ImageObserver ImageOutputStream ImageOutputStreamImpl ImageOutputStreamSpi
- ImageProducer ImageReader ImageReaderSpi ImageReaderWriterSpi ImageReadParam ImageTranscoder
- ImageTranscoderSpi ImageTypeSpecifier ImageView ImageWriteParam ImageWriter ImageWriterSpi
- ImagingOpException IncompatibleClassChangeError IncompleteAnnotationException IndexColorModel
- IndexedPropertyChangeEvent IndexedPropertyDescriptor IndexOutOfBoundsException Inet4Address Inet6Address
- InetAddress InetSocketAddress Inflater InflaterInputStream InheritableThreadLocal Inherited
- InitialContext InitialContextFactory InitialContextFactoryBuilder InitialDirContext InitialLdapContext
- InlineView InputContext InputEvent InputMap InputMapUIResource InputMethod InputMethodContext
- InputMethodDescriptor InputMethodEvent InputMethodHighlight InputMethodListener InputMethodRequests
- InputMismatchException InputStream InputStreamReader InputSubset InputVerifier Insets InsetsUIResource
- InstanceAlreadyExistsException InstanceNotFoundException InstantiationError InstantiationException
- Instrument Instrumentation InsufficientResourcesException IntBuffer Integer IntegerSyntax InternalError
- InternalFrameAdapter InternalFrameEvent InternalFrameFocusTraversalPolicy InternalFrameListener
- InternalFrameUI InternationalFormatter InterruptedException InterruptedIOException
- InterruptedNamingException InterruptibleChannel IntrospectionException Introspector
- InvalidActivityException InvalidAlgorithmParameterException InvalidApplicationException
- InvalidAttributeIdentifierException InvalidAttributesException InvalidAttributeValueException
- InvalidClassException InvalidDnDOperationException InvalidKeyException InvalidKeySpecException
- InvalidMarkException InvalidMidiDataException InvalidNameException InvalidObjectException
- InvalidOpenTypeException InvalidParameterException InvalidParameterSpecException
- InvalidPreferencesFormatException InvalidPropertiesFormatException InvalidRelationIdException
- InvalidRelationServiceException InvalidRelationTypeException InvalidRoleInfoException
- InvalidRoleValueException InvalidSearchControlsException InvalidSearchFilterException
- InvalidTargetObjectTypeException InvalidTransactionException InvocationEvent InvocationHandler
- InvocationTargetException IOException ItemEvent ItemListener ItemSelectable Iterable Iterator
- IvParameterSpec JApplet JarEntry JarException JarFile JarInputStream JarOutputStream JarURLConnection
- JButton JCheckBox JCheckBoxMenuItem JColorChooser JComboBox JComponent JdbcRowSet JDesktopPane JDialog
- JEditorPane JFileChooser JFormattedTextField JFrame JInternalFrame JLabel JLayeredPane JList JMenu
- JMenuBar JMenuItem JMException JMRuntimeException JMXAuthenticator JMXConnectionNotification
- JMXConnector JMXConnectorFactory JMXConnectorProvider JMXConnectorServer JMXConnectorServerFactory
- JMXConnectorServerMBean JMXConnectorServerProvider JMXPrincipal JMXProviderException
- JMXServerErrorException JMXServiceURL JobAttributes JobHoldUntil JobImpressions JobImpressionsCompleted
- JobImpressionsSupported JobKOctets JobKOctetsProcessed JobKOctetsSupported JobMediaSheets
- JobMediaSheetsCompleted JobMediaSheetsSupported JobMessageFromOperator JobName JobOriginatingUserName
- JobPriority JobPrioritySupported JobSheets JobState JobStateReason JobStateReasons Joinable JoinRowSet
- JOptionPane JPanel JPasswordField JPEGHuffmanTable JPEGImageReadParam JPEGImageWriteParam JPEGQTable
- JPopupMenu JProgressBar JRadioButton JRadioButtonMenuItem JRootPane JScrollBar JScrollPane JSeparator
- JSlider JSpinner JSplitPane JTabbedPane JTable JTableHeader JTextArea JTextComponent JTextField
- JTextPane JToggleButton JToolBar JToolTip JTree JViewport JWindow KerberosKey KerberosPrincipal
- KerberosTicket Kernel Key KeyAdapter KeyAgreement KeyAgreementSpi KeyAlreadyExistsException
- KeyboardFocusManager KeyEvent KeyEventDispatcher KeyEventPostProcessor KeyException KeyFactory
- KeyFactorySpi KeyGenerator KeyGeneratorSpi KeyListener KeyManagementException KeyManager
- KeyManagerFactory KeyManagerFactorySpi Keymap KeyPair KeyPairGenerator KeyPairGeneratorSpi KeyRep
- KeySpec KeyStore KeyStoreBuilderParameters KeyStoreException KeyStoreSpi KeyStroke Label LabelUI
- LabelView LanguageCallback LastOwnerException LayeredHighlighter LayoutFocusTraversalPolicy
- LayoutManager LayoutManager2 LayoutQueue LDAPCertStoreParameters LdapContext LdapName
- LdapReferralException Lease Level LimitExceededException Line Line2D LineBorder LineBreakMeasurer
- LineEvent LineListener LineMetrics LineNumberInputStream LineNumberReader LineUnavailableException
- LinkageError LinkedBlockingQueue LinkedHashMap LinkedHashSet LinkedList LinkException LinkLoopException
- LinkRef List ListCellRenderer ListDataEvent ListDataListener ListenerNotFoundException ListIterator
- ListModel ListResourceBundle ListSelectionEvent ListSelectionListener ListSelectionModel ListUI ListView
- LoaderHandler Locale LocateRegistry Lock LockSupport Logger LoggingMXBean LoggingPermission LoginContext
- LoginException LoginModule LogManager LogRecord LogStream Long LongBuffer LookAndFeel LookupOp
- LookupTable Mac MacSpi MalformedInputException MalformedLinkException MalformedObjectNameException
- MalformedParameterizedTypeException MalformedURLException ManagementFactory ManagementPermission
- ManageReferralControl ManagerFactoryParameters Manifest Map MappedByteBuffer MarshalException
- MarshalledObject MaskFormatter Matcher MatchResult Math MathContext MatteBorder MBeanAttributeInfo
- MBeanConstructorInfo MBeanException MBeanFeatureInfo MBeanInfo MBeanNotificationInfo MBeanOperationInfo
- MBeanParameterInfo MBeanPermission MBeanRegistration MBeanRegistrationException MBeanServer
- MBeanServerBuilder MBeanServerConnection MBeanServerDelegate MBeanServerDelegateMBean MBeanServerFactory
- MBeanServerForwarder MBeanServerInvocationHandler MBeanServerNotification MBeanServerNotificationFilter
- MBeanServerPermission MBeanTrustPermission Media MediaName MediaPrintableArea MediaSize MediaSizeName
- MediaTracker MediaTray Member MemoryCacheImageInputStream MemoryCacheImageOutputStream MemoryHandler
- MemoryImageSource MemoryManagerMXBean MemoryMXBean MemoryNotificationInfo MemoryPoolMXBean MemoryType
- MemoryUsage Menu MenuBar MenuBarUI MenuComponent MenuContainer MenuDragMouseEvent MenuDragMouseListener
- MenuElement MenuEvent MenuItem MenuItemUI MenuKeyEvent MenuKeyListener MenuListener MenuSelectionManager
- MenuShortcut MessageDigest MessageDigestSpi MessageFormat MetaEventListener MetalBorders MetalButtonUI
- MetalCheckBoxIcon MetalCheckBoxUI MetalComboBoxButton MetalComboBoxEditor MetalComboBoxIcon
- MetalComboBoxUI MetalDesktopIconUI MetalFileChooserUI MetalIconFactory MetalInternalFrameTitlePane
- MetalInternalFrameUI MetalLabelUI MetalLookAndFeel MetalMenuBarUI MetalPopupMenuSeparatorUI
- MetalProgressBarUI MetalRadioButtonUI MetalRootPaneUI MetalScrollBarUI MetalScrollButton
- MetalScrollPaneUI MetalSeparatorUI MetalSliderUI MetalSplitPaneUI MetalTabbedPaneUI MetalTextFieldUI
- MetalTheme MetalToggleButtonUI MetalToolBarUI MetalToolTipUI MetalTreeUI MetaMessage Method
- MethodDescriptor MGF1ParameterSpec MidiChannel MidiDevice MidiDeviceProvider MidiEvent MidiFileFormat
- MidiFileReader MidiFileWriter MidiMessage MidiSystem MidiUnavailableException MimeTypeParseException
- MinimalHTMLWriter MissingFormatArgumentException MissingFormatWidthException MissingResourceException
- Mixer MixerProvider MLet MLetMBean ModelMBean ModelMBeanAttributeInfo ModelMBeanConstructorInfo
- ModelMBeanInfo ModelMBeanInfoSupport ModelMBeanNotificationBroadcaster ModelMBeanNotificationInfo
- ModelMBeanOperationInfo ModificationItem Modifier Monitor MonitorMBean MonitorNotification
- MonitorSettingException MouseAdapter MouseDragGestureRecognizer MouseEvent MouseInfo MouseInputAdapter
- MouseInputListener MouseListener MouseMotionAdapter MouseMotionListener MouseWheelEvent
- MouseWheelListener MultiButtonUI MulticastSocket MultiColorChooserUI MultiComboBoxUI MultiDesktopIconUI
- MultiDesktopPaneUI MultiDoc MultiDocPrintJob MultiDocPrintService MultiFileChooserUI
- MultiInternalFrameUI MultiLabelUI MultiListUI MultiLookAndFeel MultiMenuBarUI MultiMenuItemUI
- MultiOptionPaneUI MultiPanelUI MultiPixelPackedSampleModel MultipleDocumentHandling MultipleMaster
- MultiPopupMenuUI MultiProgressBarUI MultiRootPaneUI MultiScrollBarUI MultiScrollPaneUI MultiSeparatorUI
- MultiSliderUI MultiSpinnerUI MultiSplitPaneUI MultiTabbedPaneUI MultiTableHeaderUI MultiTableUI
- MultiTextUI MultiToolBarUI MultiToolTipUI MultiTreeUI MultiViewportUI MutableAttributeSet
- MutableComboBoxModel MutableTreeNode Name NameAlreadyBoundException NameCallback NameClassPair
- NameNotFoundException NameParser NamespaceChangeListener NamespaceContext Naming NamingEnumeration
- NamingEvent NamingException NamingExceptionEvent NamingListener NamingManager NamingSecurityException
- NavigationFilter NegativeArraySizeException NetPermission NetworkInterface NoClassDefFoundError
- NoConnectionPendingException NodeChangeEvent NodeChangeListener NoInitialContextException
- NoninvertibleTransformException NonReadableChannelException NonWritableChannelException
- NoPermissionException NoRouteToHostException NoSuchAlgorithmException NoSuchAttributeException
- NoSuchElementException NoSuchFieldError NoSuchFieldException NoSuchMethodError NoSuchMethodException
- NoSuchObjectException NoSuchPaddingException NoSuchProviderException NotActiveException
- NotBoundException NotCompliantMBeanException NotContextException Notification NotificationBroadcaster
- NotificationBroadcasterSupport NotificationEmitter NotificationFilter NotificationFilterSupport
- NotificationListener NotificationResult NotOwnerException NotSerializableException NotYetBoundException
- NotYetConnectedException NullCipher NullPointerException Number NumberFormat NumberFormatException
- NumberFormatter NumberOfDocuments NumberOfInterveningJobs NumberUp NumberUpSupported NumericShaper
- OAEPParameterSpec Object ObjectChangeListener ObjectFactory ObjectFactoryBuilder ObjectInput
- ObjectInputStream ObjectInputValidation ObjectInstance ObjectName ObjectOutput ObjectOutputStream
- ObjectStreamClass ObjectStreamConstants ObjectStreamException ObjectStreamField ObjectView ObjID
- Observable Observer OceanTheme OpenDataException OpenMBeanAttributeInfo OpenMBeanAttributeInfoSupport
- OpenMBeanConstructorInfo OpenMBeanConstructorInfoSupport OpenMBeanInfo OpenMBeanInfoSupport
- OpenMBeanOperationInfo OpenMBeanOperationInfoSupport OpenMBeanParameterInfo
- OpenMBeanParameterInfoSupport OpenType OperatingSystemMXBean Operation OperationNotSupportedException
- OperationsException Option OptionalDataException OptionPaneUI OrientationRequested OutOfMemoryError
- OutputDeviceAssigned OutputKeys OutputStream OutputStreamWriter OverlappingFileLockException
- OverlayLayout Override Owner Pack200 Package PackedColorModel Pageable PageAttributes
- PagedResultsControl PagedResultsResponseControl PageFormat PageRanges PagesPerMinute PagesPerMinuteColor
- Paint PaintContext PaintEvent Panel PanelUI Paper ParagraphView ParameterBlock ParameterDescriptor
- ParameterizedType ParameterMetaData ParseException ParsePosition Parser ParserConfigurationException
- ParserDelegator PartialResultException PasswordAuthentication PasswordCallback PasswordView Patch
- PathIterator Pattern PatternSyntaxException PBEKey PBEKeySpec PBEParameterSpec PDLOverrideSupported
- Permission PermissionCollection Permissions PersistenceDelegate PersistentMBean PhantomReference Pipe
- PipedInputStream PipedOutputStream PipedReader PipedWriter PixelGrabber PixelInterleavedSampleModel
- PKCS8EncodedKeySpec PKIXBuilderParameters PKIXCertPathBuilderResult PKIXCertPathChecker
- PKIXCertPathValidatorResult PKIXParameters PlainDocument PlainView Point Point2D PointerInfo Policy
- PolicyNode PolicyQualifierInfo Polygon PooledConnection Popup PopupFactory PopupMenu PopupMenuEvent
- PopupMenuListener PopupMenuUI Port PortableRemoteObject PortableRemoteObjectDelegate
- PortUnreachableException Position Predicate PreferenceChangeEvent PreferenceChangeListener Preferences
- PreferencesFactory PreparedStatement PresentationDirection Principal Printable PrinterAbortException
- PrinterException PrinterGraphics PrinterInfo PrinterIOException PrinterIsAcceptingJobs PrinterJob
- PrinterLocation PrinterMakeAndModel PrinterMessageFromOperator PrinterMoreInfo
- PrinterMoreInfoManufacturer PrinterName PrinterResolution PrinterState PrinterStateReason
- PrinterStateReasons PrinterURI PrintEvent PrintException PrintGraphics PrintJob PrintJobAdapter
- PrintJobAttribute PrintJobAttributeEvent PrintJobAttributeListener PrintJobAttributeSet PrintJobEvent
- PrintJobListener PrintQuality PrintRequestAttribute PrintRequestAttributeSet PrintService
- PrintServiceAttribute PrintServiceAttributeEvent PrintServiceAttributeListener PrintServiceAttributeSet
- PrintServiceLookup PrintStream PrintWriter PriorityBlockingQueue PriorityQueue PrivateClassLoader
- PrivateCredentialPermission PrivateKey PrivateMLet PrivilegedAction PrivilegedActionException
- PrivilegedExceptionAction Process ProcessBuilder ProfileDataException ProgressBarUI ProgressMonitor
- ProgressMonitorInputStream Properties PropertyChangeEvent PropertyChangeListener
- PropertyChangeListenerProxy PropertyChangeSupport PropertyDescriptor PropertyEditor
- PropertyEditorManager PropertyEditorSupport PropertyPermission PropertyResourceBundle
- PropertyVetoException ProtectionDomain ProtocolException Provider ProviderException Proxy ProxySelector
- PSource PSSParameterSpec PublicKey PushbackInputStream PushbackReader QName QuadCurve2D Query QueryEval
- QueryExp Queue QueuedJobCount Random RandomAccess RandomAccessFile Raster RasterFormatException RasterOp
- RC2ParameterSpec RC5ParameterSpec Rdn Readable ReadableByteChannel Reader ReadOnlyBufferException
- ReadWriteLock RealmCallback RealmChoiceCallback Receiver Rectangle Rectangle2D RectangularShape
- ReentrantLock ReentrantReadWriteLock Ref RefAddr Reference Referenceable ReferenceQueue
- ReferenceUriSchemesSupported ReferralException ReflectionException ReflectPermission Refreshable
- RefreshFailedException Region RegisterableService Registry RegistryHandler RejectedExecutionException
- RejectedExecutionHandler Relation RelationException RelationNotFoundException RelationNotification
- RelationService RelationServiceMBean RelationServiceNotRegisteredException RelationSupport
- RelationSupportMBean RelationType RelationTypeNotFoundException RelationTypeSupport Remote RemoteCall
- RemoteException RemoteObject RemoteObjectInvocationHandler RemoteRef RemoteServer RemoteStub
- RenderableImage RenderableImageOp RenderableImageProducer RenderContext RenderedImage
- RenderedImageFactory Renderer RenderingHints RepaintManager ReplicateScaleFilter RequestingUserName
- RequiredModelMBean RescaleOp ResolutionSyntax Resolver ResolveResult ResourceBundle ResponseCache Result
- ResultSet ResultSetMetaData Retention RetentionPolicy ReverbType RGBImageFilter RMIClassLoader
- RMIClassLoaderSpi RMIClientSocketFactory RMIConnection RMIConnectionImpl RMIConnectionImpl_Stub
- RMIConnector RMIConnectorServer RMIFailureHandler RMIIIOPServerImpl RMIJRMPServerImpl
- RMISecurityException RMISecurityManager RMIServer RMIServerImpl RMIServerImpl_Stub
- RMIServerSocketFactory RMISocketFactory Robot Role RoleInfo RoleInfoNotFoundException RoleList
- RoleNotFoundException RoleResult RoleStatus RoleUnresolved RoleUnresolvedList RootPaneContainer
- RootPaneUI RoundingMode RoundRectangle2D RowMapper RowSet RowSetEvent RowSetInternal RowSetListener
- RowSetMetaData RowSetMetaDataImpl RowSetReader RowSetWarning RowSetWriter RSAKey RSAKeyGenParameterSpec
- RSAMultiPrimePrivateCrtKey RSAMultiPrimePrivateCrtKeySpec RSAOtherPrimeInfo RSAPrivateCrtKey
- RSAPrivateCrtKeySpec RSAPrivateKey RSAPrivateKeySpec RSAPublicKey RSAPublicKeySpec RTFEditorKit
- RuleBasedCollator Runnable Runtime RuntimeErrorException RuntimeException RuntimeMBeanException
- RuntimeMXBean RuntimeOperationsException RuntimePermission SampleModel Sasl SaslClient SaslClientFactory
- SaslException SaslServer SaslServerFactory Savepoint SAXParser SAXParserFactory SAXResult SAXSource
- SAXTransformerFactory Scanner ScatteringByteChannel ScheduledExecutorService ScheduledFuture
- ScheduledThreadPoolExecutor Schema SchemaFactory SchemaFactoryLoader SchemaViolationException Scrollable
- Scrollbar ScrollBarUI ScrollPane ScrollPaneAdjustable ScrollPaneConstants ScrollPaneLayout ScrollPaneUI
- SealedObject SearchControls SearchResult SecretKey SecretKeyFactory SecretKeyFactorySpi SecretKeySpec
- SecureCacheResponse SecureClassLoader SecureRandom SecureRandomSpi Security SecurityException
- SecurityManager SecurityPermission Segment SelectableChannel SelectionKey Selector SelectorProvider
- Semaphore SeparatorUI Sequence SequenceInputStream Sequencer SerialArray SerialBlob SerialClob
- SerialDatalink SerialException Serializable SerializablePermission SerialJavaObject SerialRef
- SerialStruct ServerCloneException ServerError ServerException ServerNotActiveException ServerRef
- ServerRuntimeException ServerSocket ServerSocketChannel ServerSocketFactory ServiceNotFoundException
- ServicePermission ServiceRegistry ServiceUI ServiceUIFactory ServiceUnavailableException Set
- SetOfIntegerSyntax Severity Shape ShapeGraphicAttribute SheetCollate Short ShortBuffer
- ShortBufferException ShortLookupTable ShortMessage Sides Signature SignatureException SignatureSpi
- SignedObject Signer SimpleAttributeSet SimpleBeanInfo SimpleDateFormat SimpleDoc SimpleFormatter
- SimpleTimeZone SimpleType SinglePixelPackedSampleModel SingleSelectionModel Size2DSyntax
- SizeLimitExceededException SizeRequirements SizeSequence Skeleton SkeletonMismatchException
- SkeletonNotFoundException SliderUI Socket SocketAddress SocketChannel SocketException SocketFactory
- SocketHandler SocketImpl SocketImplFactory SocketOptions SocketPermission SocketSecurityException
- SocketTimeoutException SoftBevelBorder SoftReference SortControl SortedMap SortedSet
- SortingFocusTraversalPolicy SortKey SortResponseControl Soundbank SoundbankReader SoundbankResource
- Source SourceDataLine SourceLocator SpinnerDateModel SpinnerListModel SpinnerModel SpinnerNumberModel
- SpinnerUI SplitPaneUI Spring SpringLayout SQLData SQLException SQLInput SQLInputImpl SQLOutput
- SQLOutputImpl SQLPermission SQLWarning SSLContext SSLContextSpi SSLEngine SSLEngineResult SSLException
- SSLHandshakeException SSLKeyException SSLPeerUnverifiedException SSLPermission SSLProtocolException
- SslRMIClientSocketFactory SslRMIServerSocketFactory SSLServerSocket SSLServerSocketFactory SSLSession
- SSLSessionBindingEvent SSLSessionBindingListener SSLSessionContext SSLSocket SSLSocketFactory Stack
- StackOverflowError StackTraceElement StandardMBean StartTlsRequest StartTlsResponse StateEdit
- StateEditable StateFactory Statement StreamCorruptedException StreamHandler StreamPrintService
- StreamPrintServiceFactory StreamResult StreamSource StreamTokenizer StrictMath String StringBuffer
- StringBufferInputStream StringBuilder StringCharacterIterator StringContent
- StringIndexOutOfBoundsException StringMonitor StringMonitorMBean StringReader StringRefAddr
- StringSelection StringTokenizer StringValueExp StringWriter Stroke Struct Stub StubDelegate
- StubNotFoundException Style StyleConstants StyleContext StyledDocument StyledEditorKit StyleSheet
- Subject SubjectDelegationPermission SubjectDomainCombiner SupportedValuesAttribute SuppressWarnings
- SwingConstants SwingPropertyChangeSupport SwingUtilities SyncFactory SyncFactoryException
- SyncFailedException SynchronousQueue SyncProvider SyncProviderException SyncResolver SynthConstants
- SynthContext Synthesizer SynthGraphicsUtils SynthLookAndFeel SynthPainter SynthStyle SynthStyleFactory
- SysexMessage System SystemColor SystemFlavorMap TabableView TabbedPaneUI TabExpander TableCellEditor
- TableCellRenderer TableColumn TableColumnModel TableColumnModelEvent TableColumnModelListener
- TableHeaderUI TableModel TableModelEvent TableModelListener TableUI TableView TabSet TabStop TabularData
- TabularDataSupport TabularType TagElement Target TargetDataLine TargetedNotification Templates
- TemplatesHandler TextAction TextArea TextAttribute TextComponent TextEvent TextField TextHitInfo
- TextInputCallback TextLayout TextListener TextMeasurer TextOutputCallback TextSyntax TextUI TexturePaint
- Thread ThreadDeath ThreadFactory ThreadGroup ThreadInfo ThreadLocal ThreadMXBean ThreadPoolExecutor
- Throwable Tie TileObserver Time TimeLimitExceededException TimeoutException Timer
- TimerAlarmClockNotification TimerMBean TimerNotification TimerTask Timestamp TimeUnit TimeZone
- TitledBorder ToolBarUI Toolkit ToolTipManager ToolTipUI TooManyListenersException Track
- TransactionalWriter TransactionRequiredException TransactionRolledbackException Transferable
- TransferHandler TransformAttribute Transformer TransformerConfigurationException TransformerException
- TransformerFactory TransformerFactoryConfigurationError TransformerHandler Transmitter Transparency
- TreeCellEditor TreeCellRenderer TreeExpansionEvent TreeExpansionListener TreeMap TreeModel
- TreeModelEvent TreeModelListener TreeNode TreePath TreeSelectionEvent TreeSelectionListener
- TreeSelectionModel TreeSet TreeUI TreeWillExpandListener TrustAnchor TrustManager TrustManagerFactory
- TrustManagerFactorySpi Type TypeInfoProvider TypeNotPresentException Types TypeVariable UID UIDefaults
- UIManager UIResource UndeclaredThrowableException UndoableEdit UndoableEditEvent UndoableEditListener
- UndoableEditSupport UndoManager UnexpectedException UnicastRemoteObject UnknownError
- UnknownFormatConversionException UnknownFormatFlagsException UnknownGroupException UnknownHostException
- UnknownObjectException UnknownServiceException UnmappableCharacterException UnmarshalException
- UnmodifiableClassException UnmodifiableSetException UnrecoverableEntryException
- UnrecoverableKeyException Unreferenced UnresolvedAddressException UnresolvedPermission
- UnsatisfiedLinkError UnsolicitedNotification UnsolicitedNotificationEvent
- UnsolicitedNotificationListener UnsupportedAddressTypeException UnsupportedAudioFileException
- UnsupportedCallbackException UnsupportedCharsetException UnsupportedClassVersionError
- UnsupportedEncodingException UnsupportedFlavorException UnsupportedLookAndFeelException
- UnsupportedOperationException URI URIException URIResolver URISyntax URISyntaxException URL
- URLClassLoader URLConnection URLDecoder URLEncoder URLStreamHandler URLStreamHandlerFactory
- UTFDataFormatException Util UtilDelegate Utilities UUID Validator ValidatorHandler ValueExp ValueHandler
- ValueHandlerMultiFormat VariableHeightLayoutCache Vector VerifyError VetoableChangeListener
- VetoableChangeListenerProxy VetoableChangeSupport View ViewFactory ViewportLayout ViewportUI
- VirtualMachineError Visibility VMID VoiceStatus Void VolatileImage WeakHashMap WeakReference WebRowSet
- WildcardType Window WindowAdapter WindowConstants WindowEvent WindowFocusListener WindowListener
- WindowStateListener WrappedPlainView WritableByteChannel WritableRaster WritableRenderedImage
- WriteAbortedException Writer X500Principal X500PrivateCredential X509Certificate X509CertSelector
- X509CRL X509CRLEntry X509CRLSelector X509EncodedKeySpec X509ExtendedKeyManager X509Extension
- X509KeyManager X509TrustManager XAConnection XADataSource XAException XAResource Xid XMLConstants
- XMLDecoder XMLEncoder XMLFormatter XMLGregorianCalendar XMLParseException XmlReader XmlWriter XPath
- XPathConstants XPathException XPathExpression XPathExpressionException XPathFactory
- XPathFactoryConfigurationException XPathFunction XPathFunctionException XPathFunctionResolver
- XPathVariableResolver ZipEntry ZipException ZipFile ZipInputStream ZipOutputStream ZoneView
- ]
-
- end
-
-end
-end
+++ /dev/null
-module CodeRay
-module Scanners
-
- class JavaScript < Scanner
-
- include Streamable
-
- register_for :java_script
- file_extension 'js'
-
- # The actual JavaScript keywords.
- KEYWORDS = %w[
- break case catch continue default delete do else
- finally for function if in instanceof new
- return switch throw try typeof var void while with
- ]
- PREDEFINED_CONSTANTS = %w[
- false null true undefined
- ]
-
- MAGIC_VARIABLES = %w[ this arguments ] # arguments was introduced in JavaScript 1.4
-
- KEYWORDS_EXPECTING_VALUE = WordList.new.add %w[
- case delete in instanceof new return throw typeof while with
- ]
-
- # Reserved for future use.
- RESERVED_WORDS = %w[
- abstract boolean byte char class debugger double enum export extends
- final float goto implements import int interface long native package
- private protected public short static super synchronized throws transient
- volatile
- ]
-
- IDENT_KIND = WordList.new(:ident).
- add(RESERVED_WORDS, :reserved).
- add(PREDEFINED_CONSTANTS, :pre_constant).
- add(MAGIC_VARIABLES, :local_variable).
- add(KEYWORDS, :keyword)
-
- ESCAPE = / [bfnrtv\n\\'"] | x[a-fA-F0-9]{1,2} | [0-7]{1,3} /x
- UNICODE_ESCAPE = / u[a-fA-F0-9]{4} | U[a-fA-F0-9]{8} /x
- REGEXP_ESCAPE = / [bBdDsSwW] /x
- STRING_CONTENT_PATTERN = {
- "'" => /[^\\']+/,
- '"' => /[^\\"]+/,
- '/' => /[^\\\/]+/,
- }
- KEY_CHECK_PATTERN = {
- "'" => / [^\\']* (?: \\.? [^\\']* )* '? \s* : /x,
- '"' => / [^\\"]* (?: \\.? [^\\"]* )* "? \s* : /x,
- }
-
- def scan_tokens tokens, options
-
- state = :initial
- string_delimiter = nil
- value_expected = true
- key_expected = false
- function_expected = false
-
- until eos?
-
- kind = nil
- match = nil
-
- case state
-
- when :initial
-
- if match = scan(/ \s+ | \\\n /x)
- value_expected = true if !value_expected && match.index(?\n)
- tokens << [match, :space]
- next
-
- elsif scan(%r! // [^\n\\]* (?: \\. [^\n\\]* )* | /\* (?: .*? \*/ | .* ) !mx)
- value_expected = true
- kind = :comment
-
- elsif check(/\.?\d/)
- key_expected = value_expected = false
- if scan(/0[xX][0-9A-Fa-f]+/)
- kind = :hex
- elsif scan(/(?>0[0-7]+)(?![89.eEfF])/)
- kind = :oct
- elsif scan(/\d+[fF]|\d*\.\d+(?:[eE][+-]?\d+)?[fF]?|\d+[eE][+-]?\d+[fF]?/)
- kind = :float
- elsif scan(/\d+/)
- kind = :integer
- end
-
- elsif value_expected && match = scan(/<([[:alpha:]]\w*) (?: [^\/>]*\/> | .*?<\/\1>)/xim)
- html_scanner.tokenize match
- value_expected = false
- next
-
- elsif match = scan(/ [-+*=<>?:;,!&^|(\[{~%]+ | \.(?!\d) /x)
- value_expected = true
- last_operator = match[-1]
- key_expected = (last_operator == ?{) || (last_operator == ?,)
- function_expected = false
- kind = :operator
-
- elsif scan(/ [)\]}]+ /x)
- function_expected = key_expected = value_expected = false
- kind = :operator
-
- elsif match = scan(/ [$a-zA-Z_][A-Za-z_0-9$]* /x)
- kind = IDENT_KIND[match]
- value_expected = (kind == :keyword) && KEYWORDS_EXPECTING_VALUE[match]
- if kind == :ident
- if match.index(?$) # $ allowed inside an identifier
- kind = :predefined
- elsif function_expected
- kind = :function
- elsif check(/\s*[=:]\s*function\b/)
- kind = :function
- elsif key_expected && check(/\s*:/)
- kind = :key
- end
- end
- function_expected = (kind == :keyword) && (match == 'function')
- key_expected = false
-
- elsif match = scan(/["']/)
- if key_expected && check(KEY_CHECK_PATTERN[match])
- state = :key
- else
- state = :string
- end
- tokens << [:open, state]
- string_delimiter = match
- kind = :delimiter
-
- elsif value_expected && (match = scan(/\/(?=\S)/))
- tokens << [:open, :regexp]
- state = :regexp
- string_delimiter = '/'
- kind = :delimiter
-
- elsif scan(/ \/ /x)
- value_expected = true
- key_expected = false
- kind = :operator
-
- else
- getch
- kind = :error
-
- end
-
- when :string, :regexp, :key
- if scan(STRING_CONTENT_PATTERN[string_delimiter])
- kind = :content
- elsif match = scan(/["'\/]/)
- tokens << [match, :delimiter]
- if state == :regexp
- modifiers = scan(/[gim]+/)
- tokens << [modifiers, :modifier] if modifiers && !modifiers.empty?
- end
- tokens << [:close, state]
- string_delimiter = nil
- key_expected = value_expected = false
- state = :initial
- next
- elsif state != :regexp && (match = scan(/ \\ (?: #{ESCAPE} | #{UNICODE_ESCAPE} ) /mox))
- if string_delimiter == "'" && !(match == "\\\\" || match == "\\'")
- kind = :content
- else
- kind = :char
- end
- elsif state == :regexp && scan(/ \\ (?: #{ESCAPE} | #{REGEXP_ESCAPE} | #{UNICODE_ESCAPE} ) /mox)
- kind = :char
- elsif scan(/\\./m)
- kind = :content
- elsif scan(/ \\ | $ /x)
- tokens << [:close, state]
- kind = :error
- key_expected = value_expected = false
- state = :initial
- else
- raise_inspect "else case \" reached; %p not handled." % peek(1), tokens
- end
-
- else
- raise_inspect 'Unknown state', tokens
-
- end
-
- match ||= matched
- if $DEBUG and not kind
- raise_inspect 'Error token %p in line %d' %
- [[match, kind], line], tokens
- end
- raise_inspect 'Empty token', tokens unless match
-
- tokens << [match, kind]
-
- end
-
- if [:string, :regexp].include? state
- tokens << [:close, state]
- end
-
- tokens
- end
-
- protected
-
- def reset_instance
- super
- @html_scanner.reset if defined? @html_scanner
- end
-
- def html_scanner
- @html_scanner ||= CodeRay.scanner :html, :tokens => @tokens, :keep_tokens => true, :keep_state => true
- end
-
- end
-
-end
-end
+++ /dev/null
-module CodeRay
-module Scanners
-
- class JSON < Scanner
-
- include Streamable
-
- register_for :json
- file_extension 'json'
-
- KINDS_NOT_LOC = [
- :float, :char, :content, :delimiter,
- :error, :integer, :operator, :value,
- ]
-
- CONSTANTS = %w( true false null )
- IDENT_KIND = WordList.new(:key).add(CONSTANTS, :value)
-
- ESCAPE = / [bfnrt\\"\/] /x
- UNICODE_ESCAPE = / u[a-fA-F0-9]{4} /x
-
- def scan_tokens tokens, options
-
- state = :initial
- stack = []
- string_delimiter = nil
- key_expected = false
-
- until eos?
-
- kind = nil
- match = nil
-
- case state
-
- when :initial
- if match = scan(/ \s+ | \\\n /x)
- tokens << [match, :space]
- next
- elsif match = scan(/ [:,\[{\]}] /x)
- kind = :operator
- case match
- when '{' then stack << :object; key_expected = true
- when '[' then stack << :array
- when ':' then key_expected = false
- when ',' then key_expected = true if stack.last == :object
- when '}', ']' then stack.pop # no error recovery, but works for valid JSON
- end
- elsif match = scan(/ true | false | null /x)
- kind = IDENT_KIND[match]
- elsif match = scan(/-?(?:0|[1-9]\d*)/)
- kind = :integer
- if scan(/\.\d+(?:[eE][-+]?\d+)?|[eE][-+]?\d+/)
- match << matched
- kind = :float
- end
- elsif match = scan(/"/)
- state = key_expected ? :key : :string
- tokens << [:open, state]
- kind = :delimiter
- else
- getch
- kind = :error
- end
-
- when :string, :key
- if scan(/[^\\"]+/)
- kind = :content
- elsif scan(/"/)
- tokens << ['"', :delimiter]
- tokens << [:close, state]
- state = :initial
- next
- elsif scan(/ \\ (?: #{ESCAPE} | #{UNICODE_ESCAPE} ) /mox)
- kind = :char
- elsif scan(/\\./m)
- kind = :content
- elsif scan(/ \\ | $ /x)
- tokens << [:close, :delimiter]
- kind = :error
- state = :initial
- else
- raise_inspect "else case \" reached; %p not handled." % peek(1), tokens
- end
-
- else
- raise_inspect 'Unknown state', tokens
-
- end
-
- match ||= matched
- if $DEBUG and not kind
- raise_inspect 'Error token %p in line %d' %
- [[match, kind], line], tokens
- end
- raise_inspect 'Empty token', tokens unless match
-
- tokens << [match, kind]
-
- end
-
- if [:string, :key].include? state
- tokens << [:close, state]
- end
-
- tokens
- end
-
- end
-
-end
-end
+++ /dev/null
-module CodeRay
-module Scanners
-
- load :html
-
- # Original by Stefan Walk.
- class PHP < Scanner
-
- register_for :php
- file_extension 'php'
-
- KINDS_NOT_LOC = HTML::KINDS_NOT_LOC
-
- def setup
- @html_scanner = CodeRay.scanner :html, :tokens => @tokens, :keep_tokens => true, :keep_state => true
- end
-
- def reset_instance
- super
- @html_scanner.reset
- end
-
- module Words
-
- # according to http://www.php.net/manual/en/reserved.keywords.php
- KEYWORDS = %w[
- abstract and array as break case catch class clone const continue declare default do else elseif
- enddeclare endfor endforeach endif endswitch endwhile extends final for foreach function global
- goto if implements interface instanceof namespace new or private protected public static switch
- throw try use var while xor
- cfunction old_function
- ]
-
- TYPES = %w[ int integer float double bool boolean string array object resource ]
-
- LANGUAGE_CONSTRUCTS = %w[
- die echo empty exit eval include include_once isset list
- require require_once return print unset
- ]
-
- CLASSES = %w[ Directory stdClass __PHP_Incomplete_Class exception php_user_filter Closure ]
-
- # according to http://php.net/quickref.php on 2009-04-21;
- # all functions with _ excluded (module functions) and selected additional functions
- BUILTIN_FUNCTIONS = %w[
- abs acos acosh addcslashes addslashes aggregate array arsort ascii2ebcdic asin asinh asort assert atan atan2
- atanh basename bcadd bccomp bcdiv bcmod bcmul bcpow bcpowmod bcscale bcsqrt bcsub bin2hex bindec
- bindtextdomain bzclose bzcompress bzdecompress bzerrno bzerror bzerrstr bzflush bzopen bzread bzwrite
- calculhmac ceil chdir checkdate checkdnsrr chgrp chmod chop chown chr chroot clearstatcache closedir closelog
- compact constant copy cos cosh count crc32 crypt current date dcgettext dcngettext deaggregate decbin dechex
- decoct define defined deg2rad delete dgettext die dirname diskfreespace dl dngettext doubleval each
- ebcdic2ascii echo empty end ereg eregi escapeshellarg escapeshellcmd eval exec exit exp explode expm1 extract
- fclose feof fflush fgetc fgetcsv fgets fgetss file fileatime filectime filegroup fileinode filemtime fileowner
- fileperms filepro filesize filetype floatval flock floor flush fmod fnmatch fopen fpassthru fprintf fputcsv
- fputs fread frenchtojd fscanf fseek fsockopen fstat ftell ftok ftruncate fwrite getallheaders getcwd getdate
- getenv gethostbyaddr gethostbyname gethostbynamel getimagesize getlastmod getmxrr getmygid getmyinode getmypid
- getmyuid getopt getprotobyname getprotobynumber getrandmax getrusage getservbyname getservbyport gettext
- gettimeofday gettype glob gmdate gmmktime gmstrftime gregoriantojd gzclose gzcompress gzdecode gzdeflate
- gzencode gzeof gzfile gzgetc gzgets gzgetss gzinflate gzopen gzpassthru gzputs gzread gzrewind gzseek gztell
- gzuncompress gzwrite hash header hebrev hebrevc hexdec htmlentities htmlspecialchars hypot iconv idate
- implode include intval ip2long iptcembed iptcparse isset
- jddayofweek jdmonthname jdtofrench jdtogregorian jdtojewish jdtojulian jdtounix jewishtojd join jpeg2wbmp
- juliantojd key krsort ksort lcfirst lchgrp lchown levenshtein link linkinfo list localeconv localtime log
- log10 log1p long2ip lstat ltrim mail main max md5 metaphone mhash microtime min mkdir mktime msql natcasesort
- natsort next ngettext nl2br nthmac octdec opendir openlog
- ord overload pack passthru pathinfo pclose pfsockopen phpcredits phpinfo phpversion pi png2wbmp popen pos pow
- prev print printf putenv quotemeta rad2deg rand range rawurldecode rawurlencode readdir readfile readgzfile
- readline readlink realpath recode rename require reset rewind rewinddir rmdir round rsort rtrim scandir
- serialize setcookie setlocale setrawcookie settype sha1 shuffle signeurlpaiement sin sinh sizeof sleep snmpget
- snmpgetnext snmprealwalk snmpset snmpwalk snmpwalkoid sort soundex split spliti sprintf sqrt srand sscanf stat
- strcasecmp strchr strcmp strcoll strcspn strftime stripcslashes stripos stripslashes stristr strlen
- strnatcasecmp strnatcmp strncasecmp strncmp strpbrk strpos strptime strrchr strrev strripos strrpos strspn
- strstr strtok strtolower strtotime strtoupper strtr strval substr symlink syslog system tan tanh tempnam
- textdomain time tmpfile touch trim uasort ucfirst ucwords uksort umask uniqid unixtojd unlink unpack
- unserialize unset urldecode urlencode usleep usort vfprintf virtual vprintf vsprintf wordwrap
- array_change_key_case array_chunk array_combine array_count_values array_diff array_diff_assoc
- array_diff_key array_diff_uassoc array_diff_ukey array_fill array_fill_keys array_filter array_flip
- array_intersect array_intersect_assoc array_intersect_key array_intersect_uassoc array_intersect_ukey
- array_key_exists array_keys array_map array_merge array_merge_recursive array_multisort array_pad
- array_pop array_product array_push array_rand array_reduce array_reverse array_search array_shift
- array_slice array_splice array_sum array_udiff array_udiff_assoc array_udiff_uassoc array_uintersect
- array_uintersect_assoc array_uintersect_uassoc array_unique array_unshift array_values array_walk
- array_walk_recursive
- assert_options base_convert base64_decode base64_encode
- chunk_split class_exists class_implements class_parents
- count_chars debug_backtrace debug_print_backtrace debug_zval_dump
- error_get_last error_log error_reporting extension_loaded
- file_exists file_get_contents file_put_contents load_file
- func_get_arg func_get_args func_num_args function_exists
- get_browser get_called_class get_cfg_var get_class get_class_methods get_class_vars
- get_current_user get_declared_classes get_declared_interfaces get_defined_constants
- get_defined_functions get_defined_vars get_extension_funcs get_headers get_html_translation_table
- get_include_path get_included_files get_loaded_extensions get_magic_quotes_gpc get_magic_quotes_runtime
- get_meta_tags get_object_vars get_parent_class get_required_filesget_resource_type
- gc_collect_cycles gc_disable gc_enable gc_enabled
- halt_compiler headers_list headers_sent highlight_file highlight_string
- html_entity_decode htmlspecialchars_decode
- in_array include_once inclued_get_data
- is_a is_array is_binary is_bool is_buffer is_callable is_dir is_double is_executable is_file is_finite
- is_float is_infinite is_int is_integer is_link is_long is_nan is_null is_numeric is_object is_readable
- is_real is_resource is_scalar is_soap_fault is_string is_subclass_of is_unicode is_uploaded_file
- is_writable is_writeable
- locale_get_default locale_set_default
- number_format override_function parse_str parse_url
- php_check_syntax php_ini_loaded_file php_ini_scanned_files php_logo_guid php_sapi_name
- php_strip_whitespace php_uname
- preg_filter preg_grep preg_last_error preg_match preg_match_all preg_quote preg_replace
- preg_replace_callback preg_split print_r
- require_once register_shutdown_function register_tick_function
- set_error_handler set_exception_handler set_file_buffer set_include_path
- set_magic_quotes_runtime set_time_limit shell_exec
- str_getcsv str_ireplace str_pad str_repeat str_replace str_rot13 str_shuffle str_split str_word_count
- strip_tags substr_compare substr_count substr_replace
- time_nanosleep time_sleep_until
- token_get_all token_name trigger_error
- unregister_tick_function use_soap_error_handler user_error
- utf8_decode utf8_encode var_dump var_export
- version_compare
- zend_logo_guid zend_thread_id zend_version
- ]
- # TODO: more built-in PHP functions?
-
- EXCEPTIONS = %w[
- E_ERROR E_WARNING E_PARSE E_NOTICE E_CORE_ERROR E_CORE_WARNING E_COMPILE_ERROR E_COMPILE_WARNING
- E_USER_ERROR E_USER_WARNING E_USER_NOTICE E_DEPRECATED E_USER_DEPRECATED E_ALL E_STRICT
- ]
-
- CONSTANTS = %w[
- null true false self parent
- __LINE__ __DIR__ __FILE__ __LINE__
- __CLASS__ __NAMESPACE__ __METHOD__ __FUNCTION__
- PHP_VERSION PHP_MAJOR_VERSION PHP_MINOR_VERSION PHP_RELEASE_VERSION PHP_VERSION_ID PHP_EXTRA_VERSION PHP_ZTS
- PHP_DEBUG PHP_MAXPATHLEN PHP_OS PHP_SAPI PHP_EOL PHP_INT_MAX PHP_INT_SIZE DEFAULT_INCLUDE_PATH
- PEAR_INSTALL_DIR PEAR_EXTENSION_DIR PHP_EXTENSION_DIR PHP_PREFIX PHP_BINDIR PHP_LIBDIR PHP_DATADIR
- PHP_SYSCONFDIR PHP_LOCALSTATEDIR PHP_CONFIG_FILE_PATH PHP_CONFIG_FILE_SCAN_DIR PHP_SHLIB_SUFFIX
- PHP_OUTPUT_HANDLER_START PHP_OUTPUT_HANDLER_CONT PHP_OUTPUT_HANDLER_END
- __COMPILER_HALT_OFFSET__
- EXTR_OVERWRITE EXTR_SKIP EXTR_PREFIX_SAME EXTR_PREFIX_ALL EXTR_PREFIX_INVALID EXTR_PREFIX_IF_EXISTS
- EXTR_IF_EXISTS SORT_ASC SORT_DESC SORT_REGULAR SORT_NUMERIC SORT_STRING CASE_LOWER CASE_UPPER COUNT_NORMAL
- COUNT_RECURSIVE ASSERT_ACTIVE ASSERT_CALLBACK ASSERT_BAIL ASSERT_WARNING ASSERT_QUIET_EVAL CONNECTION_ABORTED
- CONNECTION_NORMAL CONNECTION_TIMEOUT INI_USER INI_PERDIR INI_SYSTEM INI_ALL M_E M_LOG2E M_LOG10E M_LN2 M_LN10
- M_PI M_PI_2 M_PI_4 M_1_PI M_2_PI M_2_SQRTPI M_SQRT2 M_SQRT1_2 CRYPT_SALT_LENGTH CRYPT_STD_DES CRYPT_EXT_DES
- CRYPT_MD5 CRYPT_BLOWFISH DIRECTORY_SEPARATOR SEEK_SET SEEK_CUR SEEK_END LOCK_SH LOCK_EX LOCK_UN LOCK_NB
- HTML_SPECIALCHARS HTML_ENTITIES ENT_COMPAT ENT_QUOTES ENT_NOQUOTES INFO_GENERAL INFO_CREDITS
- INFO_CONFIGURATION INFO_MODULES INFO_ENVIRONMENT INFO_VARIABLES INFO_LICENSE INFO_ALL CREDITS_GROUP
- CREDITS_GENERAL CREDITS_SAPI CREDITS_MODULES CREDITS_DOCS CREDITS_FULLPAGE CREDITS_QA CREDITS_ALL STR_PAD_LEFT
- STR_PAD_RIGHT STR_PAD_BOTH PATHINFO_DIRNAME PATHINFO_BASENAME PATHINFO_EXTENSION PATH_SEPARATOR CHAR_MAX
- LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_ALL LC_MESSAGES ABDAY_1 ABDAY_2 ABDAY_3 ABDAY_4 ABDAY_5
- ABDAY_6 ABDAY_7 DAY_1 DAY_2 DAY_3 DAY_4 DAY_5 DAY_6 DAY_7 ABMON_1 ABMON_2 ABMON_3 ABMON_4 ABMON_5 ABMON_6
- ABMON_7 ABMON_8 ABMON_9 ABMON_10 ABMON_11 ABMON_12 MON_1 MON_2 MON_3 MON_4 MON_5 MON_6 MON_7 MON_8 MON_9
- MON_10 MON_11 MON_12 AM_STR PM_STR D_T_FMT D_FMT T_FMT T_FMT_AMPM ERA ERA_YEAR ERA_D_T_FMT ERA_D_FMT ERA_T_FMT
- ALT_DIGITS INT_CURR_SYMBOL CURRENCY_SYMBOL CRNCYSTR MON_DECIMAL_POINT MON_THOUSANDS_SEP MON_GROUPING
- POSITIVE_SIGN NEGATIVE_SIGN INT_FRAC_DIGITS FRAC_DIGITS P_CS_PRECEDES P_SEP_BY_SPACE N_CS_PRECEDES
- N_SEP_BY_SPACE P_SIGN_POSN N_SIGN_POSN DECIMAL_POINT RADIXCHAR THOUSANDS_SEP THOUSEP GROUPING YESEXPR NOEXPR
- YESSTR NOSTR CODESET LOG_EMERG LOG_ALERT LOG_CRIT LOG_ERR LOG_WARNING LOG_NOTICE LOG_INFO LOG_DEBUG LOG_KERN
- LOG_USER LOG_MAIL LOG_DAEMON LOG_AUTH LOG_SYSLOG LOG_LPR LOG_NEWS LOG_UUCP LOG_CRON LOG_AUTHPRIV LOG_LOCAL0
- LOG_LOCAL1 LOG_LOCAL2 LOG_LOCAL3 LOG_LOCAL4 LOG_LOCAL5 LOG_LOCAL6 LOG_LOCAL7 LOG_PID LOG_CONS LOG_ODELAY
- LOG_NDELAY LOG_NOWAIT LOG_PERROR
- ]
-
- IDENT_KIND = CaseIgnoringWordList.new(:ident, true).
- add(KEYWORDS, :reserved).
- add(TYPES, :pre_type).
- add(LANGUAGE_CONSTRUCTS, :reserved).
- add(BUILTIN_FUNCTIONS, :predefined).
- add(CLASSES, :pre_constant).
- add(EXCEPTIONS, :exception).
- add(CONSTANTS, :pre_constant)
- end
-
- module RE
-
- PHP_START = /
- <script\s+[^>]*?language\s*=\s*"php"[^>]*?> |
- <script\s+[^>]*?language\s*=\s*'php'[^>]*?> |
- <\?php\d? |
- <\?(?!xml)
- /xi
-
- PHP_END = %r!
- </script> |
- \?>
- !xi
-
- HTML_INDICATOR = /<!DOCTYPE html|<(?:html|body|div|p)[> ]/i
-
- IDENTIFIER = /[a-z_\x7f-\xFF][a-z0-9_\x7f-\xFF]*/i
- VARIABLE = /\$#{IDENTIFIER}/
-
- OPERATOR = /
- \.(?!\d)=? | # dot that is not decimal point, string concatenation
- && | \|\| | # logic
- :: | -> | => | # scope, member, dictionary
- \+\+ | -- | # increment, decrement
- [,;?:()\[\]{}] | # simple delimiters
- [-+*\/%&|^]=? | # ordinary math, binary logic, assignment shortcuts
- [~@$] | # whatever
- [=!]=?=? | <> | # comparison and assignment
- <<=? | >>=? | [<>]=? # comparison and shift
- /x
-
- end
-
- def scan_tokens tokens, options
-
- states = [:initial]
- if match?(RE::PHP_START) || # starts with <?
- (match?(/\s*<\S/) && exist?(RE::PHP_START)) || # starts with tag and contains <?
- exist?(RE::HTML_INDICATOR)
- # is PHP inside HTML, so start with HTML
- else
- states << :php
- end
-
- # heredocdelim = nil
- delimiter = nil
-
- until eos?
-
- match = nil
- kind = nil
-
- case states.last
-
- when :initial # HTML
- if scan RE::PHP_START
- kind = :inline_delimiter
- states << :php
- else
- match = scan_until(/(?=#{RE::PHP_START})/o) || scan_until(/\z/)
- @html_scanner.tokenize match unless match.empty?
- next
- end
-
- when :php
- if scan RE::PHP_END
- kind = :inline_delimiter
- states = [:initial]
-
- elsif scan(/\s+/)
- kind = :space
-
- elsif scan(/ \/\* (?: .*? \*\/ | .* ) /mx)
- kind = :comment
-
- elsif scan(%r!(?://|#).*?(?=#{RE::PHP_END}|$)!o)
- kind = :comment
-
- elsif match = scan(RE::IDENTIFIER)
- kind = Words::IDENT_KIND[match]
- if kind == :ident && check(/:(?!:)/) #&& tokens[-2][0] == 'case'
- # FIXME: don't match a?b:c
- kind = :label
- elsif kind == :ident && match =~ /^[A-Z]/
- kind = :constant
- elsif kind == :reserved && match == 'class'
- states << :class_expected
- elsif kind == :reserved && match == 'function'
- states << :function_expected
- end
-
- elsif scan(/(?:\d+\.\d*|\d*\.\d+)(?:e[-+]?\d+)?|\d+e[-+]?\d+/i)
- kind = :float
-
- elsif scan(/0x[0-9a-fA-F]+/)
- kind = :hex
-
- elsif scan(/\d+/)
- kind = :integer
-
- elsif scan(/'/)
- tokens << [:open, :string]
- kind = :delimiter
- states.push :sqstring
-
- elsif match = scan(/["`]/)
- tokens << [:open, :string]
- delimiter = match
- kind = :delimiter
- states.push :dqstring
-
- # TODO: Heredocs
- # See http://de2.php.net/manual/en/language.types.string.php#language.types.string.syntax.heredoc
- elsif match = scan(/<<<(#{RE::IDENTIFIER})/o)
- tokens << [:open, :string]
- heredocdelim = Regexp.escape self[1]
- tokens << [match, :delimiter]
- next if eos?
- tokens << [scan_until(/\n(?=#{heredocdelim};?$)|\z/), :content]
- next if eos?
- tokens << [scan(/#{heredocdelim}/), :delimiter]
- tokens << [:close, :string]
- next
-
- elsif scan RE::VARIABLE
- kind = :local_variable
-
- elsif scan(/\{/)
- kind = :operator
- states.push :php
-
- elsif scan(/\}/)
- if states.size == 1
- kind = :error
- else
- states.pop
- if states.last.is_a?(::Array)
- delimiter = states.last[1]
- states[-1] = states.last[0]
- tokens << [matched, :delimiter]
- tokens << [:close, :inline]
- next
- else
- kind = :operator
- end
- end
-
- elsif scan(/#{RE::OPERATOR}/o)
- kind = :operator
-
- else
- getch
- kind = :error
-
- end
-
- when :sqstring
- if scan(/[^'\\]+/)
- kind = :content
- elsif scan(/'/)
- tokens << [matched, :delimiter]
- tokens << [:close, :string]
- delimiter = nil
- states.pop
- next
- elsif scan(/\\[\\'\n]/)
- kind = :char
- elsif scan(/\\./m)
- kind = :content
- elsif scan(/\\/)
- kind = :error
- end
-
- when :dqstring
- if scan(delimiter == '"' ? /[^"${\\]+/ : /[^`${\\]+/)
- kind = :content
- elsif scan(delimiter == '"' ? /"/ : /`/)
- tokens << [matched, :delimiter]
- tokens << [:close, :string]
- delimiter = nil
- states.pop
- next
- elsif scan(/\\(?:x[0-9a-fA-F]{2}|\d{3})/)
- kind = :char
- elsif scan(delimiter == '"' ? /\\["\\\nfnrtv]/ : /\\[`\\\nfnrtv]/)
- kind = :char
- elsif scan(/\\./m)
- kind = :content
- elsif scan(/\\/)
- kind = :error
- elsif match = scan(/#{RE::VARIABLE}/o)
- kind = :local_variable
- # $foo[bar] and $foo->bar kind of stuff
- # TODO: highlight tokens separately!
- if check(/\[#{RE::IDENTIFIER}\]/o)
- match << scan(/\[#{RE::IDENTIFIER}\]/o)
- elsif check(/\[/)
- match << scan(/\[#{RE::IDENTIFIER}?/o)
- kind = :error
- elsif check(/->#{RE::IDENTIFIER}/o)
- match << scan(/->#{RE::IDENTIFIER}/o)
- elsif check(/->/)
- match << scan(/->/)
- kind = :error
- end
- elsif match = scan(/\{/)
- if check(/\$/)
- kind = :delimiter
- states[-1] = [states.last, delimiter]
- delimiter = nil
- states.push :php
- tokens << [:open, :inline]
- else
- kind = :string
- end
- elsif scan(/\$\{#{RE::IDENTIFIER}\}/o)
- kind = :local_variable
- elsif scan(/\$/)
- kind = :content
- end
-
- when :class_expected
- if scan(/\s+/)
- kind = :space
- elsif match = scan(/#{RE::IDENTIFIER}/o)
- kind = :class
- states.pop
- else
- states.pop
- next
- end
-
- when :function_expected
- if scan(/\s+/)
- kind = :space
- elsif scan(/&/)
- kind = :operator
- elsif match = scan(/#{RE::IDENTIFIER}/o)
- kind = :function
- states.pop
- else
- states.pop
- next
- end
-
- else
- raise_inspect 'Unknown state!', tokens, states
- end
-
- match ||= matched
- if $DEBUG and not kind
- raise_inspect 'Error token %p in line %d' %
- [[match, kind], line], tokens, states
- end
- raise_inspect 'Empty token', tokens, states unless match
-
- tokens << [match, kind]
-
- end
-
- tokens
- end
-
- end
-
-end
-end
+++ /dev/null
-module CodeRay
-module Scanners
-
- # Bases on pygments' PythonLexer, see
- # http://dev.pocoo.org/projects/pygments/browser/pygments/lexers/agile.py.
- class Python < Scanner
-
- include Streamable
-
- register_for :python
- file_extension 'py'
-
- KEYWORDS = [
- 'and', 'as', 'assert', 'break', 'class', 'continue', 'def',
- 'del', 'elif', 'else', 'except', 'finally', 'for',
- 'from', 'global', 'if', 'import', 'in', 'is', 'lambda', 'not',
- 'or', 'pass', 'raise', 'return', 'try', 'while', 'with', 'yield',
- 'nonlocal', # new in Python 3
- ]
-
- OLD_KEYWORDS = [
- 'exec', 'print', # gone in Python 3
- ]
-
- PREDEFINED_METHODS_AND_TYPES = %w[
- __import__ abs all any apply basestring bin bool buffer
- bytearray bytes callable chr classmethod cmp coerce compile
- complex delattr dict dir divmod enumerate eval execfile exit
- file filter float frozenset getattr globals hasattr hash hex id
- input int intern isinstance issubclass iter len list locals
- long map max min next object oct open ord pow property range
- raw_input reduce reload repr reversed round set setattr slice
- sorted staticmethod str sum super tuple type unichr unicode
- vars xrange zip
- ]
-
- PREDEFINED_EXCEPTIONS = %w[
- ArithmeticError AssertionError AttributeError
- BaseException DeprecationWarning EOFError EnvironmentError
- Exception FloatingPointError FutureWarning GeneratorExit IOError
- ImportError ImportWarning IndentationError IndexError KeyError
- KeyboardInterrupt LookupError MemoryError NameError
- NotImplemented NotImplementedError OSError OverflowError
- OverflowWarning PendingDeprecationWarning ReferenceError
- RuntimeError RuntimeWarning StandardError StopIteration
- SyntaxError SyntaxWarning SystemError SystemExit TabError
- TypeError UnboundLocalError UnicodeDecodeError
- UnicodeEncodeError UnicodeError UnicodeTranslateError
- UnicodeWarning UserWarning ValueError Warning ZeroDivisionError
- ]
-
- PREDEFINED_VARIABLES_AND_CONSTANTS = [
- 'False', 'True', 'None', # "keywords" since Python 3
- 'self', 'Ellipsis', 'NotImplemented',
- ]
-
- IDENT_KIND = WordList.new(:ident).
- add(KEYWORDS, :keyword).
- add(OLD_KEYWORDS, :old_keyword).
- add(PREDEFINED_METHODS_AND_TYPES, :predefined).
- add(PREDEFINED_VARIABLES_AND_CONSTANTS, :pre_constant).
- add(PREDEFINED_EXCEPTIONS, :exception)
-
- NAME = / [^\W\d] \w* /x
- ESCAPE = / [abfnrtv\n\\'"] | x[a-fA-F0-9]{1,2} | [0-7]{1,3} /x
- UNICODE_ESCAPE = / u[a-fA-F0-9]{4} | U[a-fA-F0-9]{8} | N\{[-\w ]+\} /x
-
- OPERATOR = /
- \.\.\. | # ellipsis
- \.(?!\d) | # dot but not decimal point
- [,;:()\[\]{}] | # simple delimiters
- \/\/=? | \*\*=? | # special math
- [-+*\/%&|^]=? | # ordinary math and binary logic
- [~`] | # binary complement and inspection
- <<=? | >>=? | [<>=]=? | != # comparison and assignment
- /x
-
- STRING_DELIMITER_REGEXP = Hash.new do |h, delimiter|
- h[delimiter] = Regexp.union delimiter
- end
-
- STRING_CONTENT_REGEXP = Hash.new do |h, delimiter|
- h[delimiter] = / [^\\\n]+? (?= \\ | $ | #{Regexp.escape(delimiter)} ) /x
- end
-
- DEF_NEW_STATE = WordList.new(:initial).
- add(%w(def), :def_expected).
- add(%w(import from), :include_expected).
- add(%w(class), :class_expected)
-
- DESCRIPTOR = /
- #{NAME}
- (?: \. #{NAME} )*
- | \*
- /x
-
- def scan_tokens tokens, options
-
- state = :initial
- string_delimiter = nil
- string_raw = false
- import_clause = class_name_follows = last_token_dot = false
- unicode = string.respond_to?(:encoding) && string.encoding.name == 'UTF-8'
- from_import_state = []
-
- until eos?
-
- kind = nil
- match = nil
-
- if state == :string
- if scan(STRING_DELIMITER_REGEXP[string_delimiter])
- tokens << [matched, :delimiter]
- tokens << [:close, :string]
- state = :initial
- next
- elsif string_delimiter.size == 3 && scan(/\n/)
- kind = :content
- elsif scan(STRING_CONTENT_REGEXP[string_delimiter])
- kind = :content
- elsif !string_raw && scan(/ \\ #{ESCAPE} /ox)
- kind = :char
- elsif scan(/ \\ #{UNICODE_ESCAPE} /ox)
- kind = :char
- elsif scan(/ \\ . /x)
- kind = :content
- elsif scan(/ \\ | $ /x)
- tokens << [:close, :string]
- kind = :error
- state = :initial
- else
- raise_inspect "else case \" reached; %p not handled." % peek(1), tokens, state
- end
-
- elsif match = scan(/ [ \t]+ | \\\n /x)
- tokens << [match, :space]
- next
-
- elsif match = scan(/\n/)
- tokens << [match, :space]
- state = :initial if state == :include_expected
- next
-
- elsif match = scan(/ \# [^\n]* /mx)
- tokens << [match, :comment]
- next
-
- elsif state == :initial
-
- if scan(/#{OPERATOR}/o)
- kind = :operator
-
- elsif match = scan(/(u?r?|b)?("""|"|'''|')/i)
- tokens << [:open, :string]
- string_delimiter = self[2]
- string_raw = false
- modifiers = self[1]
- unless modifiers.empty?
- string_raw = !!modifiers.index(?r)
- tokens << [modifiers, :modifier]
- match = string_delimiter
- end
- state = :string
- kind = :delimiter
-
- # TODO: backticks
-
- elsif match = scan(unicode ? /#{NAME}/uo : /#{NAME}/o)
- kind = IDENT_KIND[match]
- # TODO: keyword arguments
- kind = :ident if last_token_dot
- if kind == :old_keyword
- kind = check(/\(/) ? :ident : :keyword
- elsif kind == :predefined && check(/ *=/)
- kind = :ident
- elsif kind == :keyword
- state = DEF_NEW_STATE[match]
- from_import_state << match.to_sym if state == :include_expected
- end
-
- elsif scan(/@[a-zA-Z0-9_.]+[lL]?/)
- kind = :decorator
-
- elsif scan(/0[xX][0-9A-Fa-f]+[lL]?/)
- kind = :hex
-
- elsif scan(/0[bB][01]+[lL]?/)
- kind = :bin
-
- elsif match = scan(/(?:\d*\.\d+|\d+\.\d*)(?:[eE][+-]?\d+)?|\d+[eE][+-]?\d+/)
- kind = :float
- if scan(/[jJ]/)
- match << matched
- kind = :imaginary
- end
-
- elsif scan(/0[oO][0-7]+|0[0-7]+(?![89.eE])[lL]?/)
- kind = :oct
-
- elsif match = scan(/\d+([lL])?/)
- kind = :integer
- if self[1] == nil && scan(/[jJ]/)
- match << matched
- kind = :imaginary
- end
-
- else
- getch
- kind = :error
-
- end
-
- elsif state == :def_expected
- state = :initial
- if match = scan(unicode ? /#{NAME}/uo : /#{NAME}/o)
- kind = :method
- else
- next
- end
-
- elsif state == :class_expected
- state = :initial
- if match = scan(unicode ? /#{NAME}/uo : /#{NAME}/o)
- kind = :class
- else
- next
- end
-
- elsif state == :include_expected
- if match = scan(unicode ? /#{DESCRIPTOR}/uo : /#{DESCRIPTOR}/o)
- kind = :include
- if match == 'as'
- kind = :keyword
- from_import_state << :as
- elsif from_import_state.first == :from && match == 'import'
- kind = :keyword
- from_import_state << :import
- elsif from_import_state.last == :as
- # kind = match[0,1][unicode ? /[[:upper:]]/u : /[[:upper:]]/] ? :class : :method
- kind = :ident
- from_import_state.pop
- elsif IDENT_KIND[match] == :keyword
- unscan
- match = nil
- state = :initial
- next
- end
- elsif match = scan(/,/)
- from_import_state.pop if from_import_state.last == :as
- kind = :operator
- else
- from_import_state = []
- state = :initial
- next
- end
-
- else
- raise_inspect 'Unknown state', tokens, state
-
- end
-
- match ||= matched
- if $DEBUG and not kind
- raise_inspect 'Error token %p in line %d' %
- [[match, kind], line], tokens, state
- end
- raise_inspect 'Empty token', tokens, state unless match
-
- last_token_dot = match == '.'
-
- tokens << [match, kind]
-
- end
-
- if state == :string
- tokens << [:close, :string]
- end
-
- tokens
- end
-
- end
-
-end
-end
+++ /dev/null
-module CodeRay module Scanners
-
- # by Josh Goebel
- class SQL < Scanner
-
- register_for :sql
-
- RESERVED_WORDS = %w(
- create database table index trigger drop primary key set select
- insert update delete replace into
- on from values before and or if exists case when
- then else as group order by avg where
- join inner outer union engine not
- like end using collate show columns begin
- )
-
- PREDEFINED_TYPES = %w(
- char varchar enum binary text tinytext mediumtext
- longtext blob tinyblob mediumblob longblob timestamp
- date time datetime year double decimal float int
- integer tinyint mediumint bigint smallint unsigned bit
- bool boolean hex bin oct
- )
-
- PREDEFINED_FUNCTIONS = %w( sum cast abs pi count min max avg )
-
- DIRECTIVES = %w( auto_increment unique default charset )
-
- PREDEFINED_CONSTANTS = %w( null true false )
-
- IDENT_KIND = CaseIgnoringWordList.new(:ident).
- add(RESERVED_WORDS, :reserved).
- add(PREDEFINED_TYPES, :pre_type).
- add(PREDEFINED_CONSTANTS, :pre_constant).
- add(PREDEFINED_FUNCTIONS, :predefined).
- add(DIRECTIVES, :directive)
-
- ESCAPE = / [rbfntv\n\\\/'"] | x[a-fA-F0-9]{1,2} | [0-7]{1,3} | . /mx
- UNICODE_ESCAPE = / u[a-fA-F0-9]{4} | U[a-fA-F0-9]{8} /x
-
- STRING_PREFIXES = /[xnb]|_\w+/i
-
- def scan_tokens tokens, options
-
- state = :initial
- string_type = nil
- string_content = ''
-
- until eos?
-
- kind = nil
- match = nil
-
- if state == :initial
-
- if scan(/ \s+ | \\\n /x)
- kind = :space
-
- elsif scan(/^(?:--\s?|#).*/)
- kind = :comment
-
- elsif scan(%r! /\* (?: .*? \*/ | .* ) !mx)
- kind = :comment
-
- elsif scan(/ [-+*\/=<>;,!&^|()\[\]{}~%] | \.(?!\d) /x)
- kind = :operator
-
- elsif scan(/(#{STRING_PREFIXES})?([`"'])/o)
- prefix = self[1]
- string_type = self[2]
- tokens << [:open, :string]
- tokens << [prefix, :modifier] if prefix
- match = string_type
- state = :string
- kind = :delimiter
-
- elsif match = scan(/ @? [A-Za-z_][A-Za-z_0-9]* /x)
- kind = match[0] == ?@ ? :variable : IDENT_KIND[match.downcase]
-
- elsif scan(/0[xX][0-9A-Fa-f]+/)
- kind = :hex
-
- elsif scan(/0[0-7]+(?![89.eEfF])/)
- kind = :oct
-
- elsif scan(/(?>\d+)(?![.eEfF])/)
- kind = :integer
-
- elsif scan(/\d[fF]|\d*\.\d+(?:[eE][+-]?\d+)?|\d+[eE][+-]?\d+/)
- kind = :float
-
- else
- getch
- kind = :error
-
- end
-
- elsif state == :string
- if match = scan(/[^\\"'`]+/)
- string_content << match
- next
- elsif match = scan(/["'`]/)
- if string_type == match
- if peek(1) == string_type # doubling means escape
- string_content << string_type << getch
- next
- end
- unless string_content.empty?
- tokens << [string_content, :content]
- string_content = ''
- end
- tokens << [matched, :delimiter]
- tokens << [:close, :string]
- state = :initial
- string_type = nil
- next
- else
- string_content << match
- end
- next
- elsif scan(/ \\ (?: #{ESCAPE} | #{UNICODE_ESCAPE} ) /mox)
- unless string_content.empty?
- tokens << [string_content, :content]
- string_content = ''
- end
- kind = :char
- elsif match = scan(/ \\ . /mox)
- string_content << match
- next
- elsif scan(/ \\ | $ /x)
- unless string_content.empty?
- tokens << [string_content, :content]
- string_content = ''
- end
- kind = :error
- state = :initial
- else
- raise "else case \" reached; %p not handled." % peek(1), tokens
- end
-
- else
- raise 'else-case reached', tokens
-
- end
-
- match ||= matched
- unless kind
- raise_inspect 'Error token %p in line %d' %
- [[match, kind], line], tokens, state
- end
- raise_inspect 'Empty token', tokens unless match
-
- tokens << [match, kind]
-
- end
- tokens
-
- end
-
- end
-
-end end
\ No newline at end of file
+++ /dev/null
-module CodeRay
-module Scanners
-
- # YAML Scanner
- #
- # Based on the YAML scanner from Syntax by Jamis Buck.
- class YAML < Scanner
-
- register_for :yaml
- file_extension 'yml'
-
- KINDS_NOT_LOC = :all
-
- def scan_tokens tokens, options
-
- value_expected = nil
- state = :initial
- key_indent = indent = 0
-
- until eos?
-
- kind = nil
- match = nil
-
- if bol?
- key_indent = nil
- if $DEBUG
- indent = check(/ +/) ? matched.size : 0
- tokens << [indent.to_s, :debug]
- end
- end
-
- if match = scan(/ +[\t ]*/)
- kind = :space
-
- elsif match = scan(/\n+/)
- kind = :space
- state = :initial if match.index(?\n)
-
- elsif match = scan(/#.*/)
- kind = :comment
-
- elsif bol? and case
- when match = scan(/---|\.\.\./)
- tokens << [:open, :head]
- tokens << [match, :head]
- tokens << [:close, :head]
- next
- when match = scan(/%.*/)
- tokens << [match, :doctype]
- next
- end
-
- elsif state == :value and case
- when !check(/(?:"[^"]*")(?=: |:$)/) && scan(/"/)
- tokens << [:open, :string]
- tokens << [matched, :delimiter]
- tokens << [matched, :content] if scan(/ [^"\\]* (?: \\. [^"\\]* )* /mx)
- tokens << [matched, :delimiter] if scan(/"/)
- tokens << [:close, :string]
- next
- when match = scan(/[|>][-+]?/)
- tokens << [:open, :string]
- tokens << [match, :delimiter]
- string_indent = key_indent || column(pos - match.size - 1)
- tokens << [matched, :content] if scan(/(?:\n+ {#{string_indent + 1}}.*)+/)
- tokens << [:close, :string]
- next
- when match = scan(/(?![!"*&]).+?(?=$|\s+#)/)
- tokens << [match, :string]
- string_indent = key_indent || column(pos - match.size - 1)
- tokens << [matched, :string] if scan(/(?:\n+ {#{string_indent + 1}}.*)+/)
- next
- end
-
- elsif case
- when match = scan(/[-:](?= |$)/)
- state = :value if state == :colon && (match == ':' || match == '-')
- state = :value if state == :initial && match == '-'
- kind = :operator
- when match = scan(/[,{}\[\]]/)
- kind = :operator
- when state == :initial && match = scan(/[\w.() ]*\S(?=: |:$)/)
- kind = :key
- key_indent = column(pos - match.size - 1)
- # tokens << [key_indent.inspect, :debug]
- state = :colon
- when match = scan(/(?:"[^"\n]*"|'[^'\n]*')(?=: |:$)/)
- tokens << [:open, :key]
- tokens << [match[0,1], :delimiter]
- tokens << [match[1..-2], :content]
- tokens << [match[-1,1], :delimiter]
- tokens << [:close, :key]
- key_indent = column(pos - match.size - 1)
- # tokens << [key_indent.inspect, :debug]
- state = :colon
- next
- when scan(/(![\w\/]+)(:([\w:]+))?/)
- tokens << [self[1], :type]
- if self[2]
- tokens << [':', :operator]
- tokens << [self[3], :class]
- end
- next
- when scan(/&\S+/)
- kind = :variable
- when scan(/\*\w+/)
- kind = :global_variable
- when scan(/<</)
- kind = :class_variable
- when scan(/\d\d:\d\d:\d\d/)
- kind = :oct
- when scan(/\d\d\d\d-\d\d-\d\d\s\d\d:\d\d:\d\d(\.\d+)? [-+]\d\d:\d\d/)
- kind = :oct
- when scan(/:\w+/)
- kind = :symbol
- when scan(/[^:\s]+(:(?! |$)[^:\s]*)* .*/)
- kind = :error
- when scan(/[^:\s]+(:(?! |$)[^:\s]*)*/)
- kind = :error
- end
-
- else
- getch
- kind = :error
-
- end
-
- match ||= matched
-
- raise_inspect 'Error token %p in line %d' % [[match, kind], line], tokens if $DEBUG && !kind
- raise_inspect 'Empty token', tokens unless match
-
- tokens << [match, kind]
-
- end
-
- tokens
- end
-
- end
-
-end
-end
+++ /dev/null
-# = Term::ANSIColor - ANSI escape sequences in Ruby
-#
-# == Description
-#
-# This library can be used to color/uncolor strings using ANSI escape sequences.
-#
-# == Author
-#
-# Florian Frank mailto:flori@ping.de
-#
-# == License
-#
-# This is free software; you can redistribute it and/or modify it under the
-# terms of the GNU General Public License Version 2 as published by the Free
-# Software Foundation: www.gnu.org/copyleft/gpl.html
-#
-# == Download
-#
-# The latest version of this library can be downloaded at
-#
-# * http://rubyforge.org/frs?group_id=391
-#
-# The homepage of this library is located at
-#
-# * http://term-ansicolor.rubyforge.org
-#
-# == Examples
-#
-# The file examples/example.rb in the source/gem-distribution shows how
-# this library can be used:
-# require 'term/ansicolor'
-#
-# # Use this trick to work around namespace cluttering that
-# # happens if you just include Term::ANSIColor:
-#
-# class Color
-# class << self
-# include Term::ANSIColor
-# end
-# end
-#
-# print Color.red, Color.bold, "No Namespace cluttering:", Color.clear, "\n"
-# print Color.green + "green" + Color.clear, "\n"
-# print Color.on_red(Color.green("green")), "\n"
-# print Color.yellow { Color.on_black { "yellow on_black" } }, "\n\n"
-#
-# # Or shortcut Term::ANSIColor by assignment:
-# c = Term::ANSIColor
-#
-# print c.red, c.bold, "No Namespace cluttering (alternative):", c.clear, "\n"
-# print c.green + "green" + c.clear, "\n"
-# print c.on_red(c.green("green")), "\n"
-# print c.yellow { c.on_black { "yellow on_black" } }, "\n\n"
-#
-# # Anyway, I don't define any of Term::ANSIColor's methods in this example
-# # and I want to keep it short:
-# include Term::ANSIColor
-#
-# print red, bold, "Usage as constants:", reset, "\n"
-# print clear, "clear", reset, reset, "reset", reset,
-# bold, "bold", reset, dark, "dark", reset,
-# underscore, "underscore", reset, blink, "blink", reset,
-# negative, "negative", reset, concealed, "concealed", reset, "|\n",
-# black, "black", reset, red, "red", reset, green, "green", reset,
-# yellow, "yellow", reset, blue, "blue", reset, magenta, "magenta", reset,
-# cyan, "cyan", reset, white, "white", reset, "|\n",
-# on_black, "on_black", reset, on_red, "on_red", reset,
-# on_green, "on_green", reset, on_yellow, "on_yellow", reset,
-# on_blue, "on_blue", reset, on_magenta, "on_magenta", reset,
-# on_cyan, "on_cyan", reset, on_white, "on_white", reset, "|\n\n"
-#
-# print red, bold, "Usage as unary argument methods:", reset, "\n"
-# print clear("clear"), reset("reset"), bold("bold"), dark("dark"),
-# underscore("underscore"), blink("blink"), negative("negative"),
-# concealed("concealed"), "|\n",
-# black("black"), red("red"), green("green"), yellow("yellow"),
-# blue("blue"), magenta("magenta"), cyan("cyan"), white("white"), "|\n",
-# on_black("on_black"), on_red("on_red"), on_green("on_green"),#
-# on_yellow("on_yellow"), on_blue("on_blue"), on_magenta("on_magenta"),
-# on_cyan("on_cyan"), on_white("on_white"), "|\n\n"
-#
-# print red { bold { "Usage as block forms:" } }, "\n"
-# print clear { "clear" }, reset { "reset" }, bold { "bold" },
-# dark { "dark" }, underscore { "underscore" }, blink { "blink" },
-# negative { "negative" }, concealed { "concealed" }, "|\n",
-# black { "black" }, red { "red" }, green { "green" },
-# yellow { "yellow" }, blue { "blue" }, magenta { "magenta" },
-# cyan { "cyan" }, white { "white" }, "|\n",
-# on_black { "on_black" }, on_red { "on_red" }, on_green { "on_green" },
-# on_yellow { "on_yellow" }, on_blue { "on_blue" },
-# on_magenta { "on_magenta" }, on_cyan { "on_cyan" },
-# on_white { "on_white" }, "|\n\n"
-#
-# # Usage as Mixin into String or its Subclasses
-# class String
-# include Term::ANSIColor
-# end
-#
-# print "Usage as String Mixins:".red.bold, "\n"
-# print "clear".clear, "reset".reset, "bold".bold, "dark".dark,
-# "underscore".underscore, "blink".blink, "negative".negative,
-# "concealed".concealed, "|\n",
-# "black".black, "red".red, "green".green, "yellow".yellow,
-# "blue".blue, "magenta".magenta, "cyan".cyan, "white".white, "|\n",
-# "on_black".on_black, "on_red".on_red, "on_green".on_green,
-# "on_yellow".on_yellow, "on_blue".on_blue, "on_magenta".on_magenta,
-# "on_cyan".on_cyan, "on_white".on_white, "|\n\n"
-#
-# symbols = Term::ANSIColor::attributes
-# print red { bold { "All supported attributes = " } },
-# blue { symbols.inspect }, "\n\n"
-#
-# print "Send symbols to strings:".send(:red).send(:bold), "\n"
-# print symbols[12, 8].map { |c| c.to_s.send(c) }, "\n\n"
-#
-# print red { bold { "Make strings monochromatic again:" } }, "\n"
-# print [ "red".red, "not red anymore".red.uncolored,
-# uncolored { "not red anymore".red }, uncolored("not red anymore".red)
-# ].map { |x| x + "\n" }
-module Term
- # The ANSIColor module can be used for namespacing and mixed into your own
- # classes.
- module ANSIColor
- # :stopdoc:
- ATTRIBUTES = [
- [ :clear , 0 ],
- [ :reset , 0 ], # synonym for :clear
- [ :bold , 1 ],
- [ :dark , 2 ],
- [ :italic , 3 ], # not widely implemented
- [ :underline , 4 ],
- [ :underscore , 4 ], # synonym for :underline
- [ :blink , 5 ],
- [ :rapid_blink , 6 ], # not widely implemented
- [ :negative , 7 ], # no reverse because of String#reverse
- [ :concealed , 8 ],
- [ :strikethrough, 9 ], # not widely implemented
- [ :black , 30 ],
- [ :red , 31 ],
- [ :green , 32 ],
- [ :yellow , 33 ],
- [ :blue , 34 ],
- [ :magenta , 35 ],
- [ :cyan , 36 ],
- [ :white , 37 ],
- [ :on_black , 40 ],
- [ :on_red , 41 ],
- [ :on_green , 42 ],
- [ :on_yellow , 43 ],
- [ :on_blue , 44 ],
- [ :on_magenta , 45 ],
- [ :on_cyan , 46 ],
- [ :on_white , 47 ],
- ]
-
- ATTRIBUTE_NAMES = ATTRIBUTES.transpose.first
- # :startdoc:
-
- # Returns true, if the coloring function of this module
- # is switched on, false otherwise.
- def self.coloring?
- @coloring
- end
-
- # Turns the coloring on or off globally, so you can easily do
- # this for example:
- # Term::ANSIColor::coloring = STDOUT.isatty
- def self.coloring=(val)
- @coloring = val
- end
- self.coloring = true
-
- ATTRIBUTES.each do |c, v|
- eval %Q{
- def #{c}(string = nil)
- result = ''
- result << "\e[#{v}m" if Term::ANSIColor.coloring?
- if block_given?
- result << yield
- elsif string
- result << string
- elsif respond_to?(:to_str)
- result << self
- else
- return result #only switch on
- end
- result << "\e[0m" if Term::ANSIColor.coloring?
- result
- end
- }
- end
-
- # Regular expression that is used to scan for ANSI-sequences while
- # uncoloring strings.
- COLORED_REGEXP = /\e\[([34][0-7]|[0-9])m/
-
- # Returns an uncolored version of the string, that is all
- # ANSI-sequences are stripped from the string.
- def uncolored(string = nil) # :yields:
- if block_given?
- yield.gsub(COLORED_REGEXP, '')
- elsif string
- string.gsub(COLORED_REGEXP, '')
- elsif respond_to?(:to_str)
- gsub(COLORED_REGEXP, '')
- else
- ''
- end
- end
-
- module_function
-
- # Returns an array of all Term::ANSIColor attributes as symbols.
- def attributes
- ATTRIBUTE_NAMES
- end
- extend self
- end
-end
- # vim: set et sw=2 ts=2:
OSDN Git Service
